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encode  human  knowledge  to  solve  classical  planning  problems  faster,  HTN  planning 
is  more  expressive  than  classical  planning,  even  to  the  point  of  being  undecidable 
in  the  general  case.  However,  HTN  planning  is  not  just  a  way  to  solve  planning 

problems  faster,  but  is  itself  a  search  problem  that  can  bene%  from  its  own  distinct  search  algorithms  and  heuristics. 

The  dissertation  examines  the  complexities  of  various  HTN  planning  problem 
classes  in  order  to  motivate  the  development  of  heuristic  search  algorithms  for  HTN 
planning  which  are  guaranteed  to  tenninate  on  a  large  class  of  syntactically  identifiable 
problems,  as  well  as  domain  independent  heuristics  for  those  algorithms  to 
use.  This  will  allow  HTN  planning  to  be  used  in  a  number  of  areas  where  problems 
may  be  unsolvable,  including  during  the  initial  development  of  a  domain  and  for  use 
in  policy  generation  in  non-deterministic  planning  environments. 

In  particular,  this  dissertation  analyzes  two  commonly  used  algorithms  for 
HTN  planning  and  describes  the  subsets  of  HTN  problems  that  these  algorithms 
tenninate  on.  This  allows  us  to  discuss  the  run-times  of  these  algorithms  and  compare 
the  expressivity  of  the  classes  of  problems  they  decide.  We  provide  two  new 
HTN  algorithms  which  terminate  on  a  strictly  broader  and  more  expressive  set  of 
HTN  problems. 

We  also  analyze  the  complexity  of  delete-free  HTN  planning,  an  analogue  to 
delete-free  classical  planning  which  is  the  base  of  many  classical  planning  heuristics. 

We  show  that  delete-free  HTN  planning  is  NP-complete,  putting  the  existence  of 
strict-semantics  delete-relaxation-based  HTN  heuristics  out  of  reach  for  practical 
purposes. 

Finally,  we  provide  a  translation  of  a  large  subset  of  HTN  planning  to  classical 
planning,  which  allows  us  to  use  a  classical  planner  as  a  surrogate  for  a  heuristic 
HTN  planner.  Our  experiments  show  that  even  small  amounts  and  incomplete 
amounts  of  HTN  knowledge,  when  translated  into  PDDL  using  our  algorithm,  can 
greatly  improve  a  classical  planner's  performance. 
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Chapter  1:  Introduction 


Despite  being  more  expressive  than  classical  planning,  Hierarchical  Task  Net¬ 
work  (HTN)  planning  is  often  viewed  as  just  a  way  to  encode  human  knowledge 
to  solve  classical  planning  problems  faster.  The  last  decade  has  seen  a  number  of 
advancements  in  classical  planning  that  reduce  planning  time,  increase  plan  quality, 
and  generally  broaden  classical  planning’s  applicability.  However,  many  of  these 
techniques  to  not  directly  transfer,  since  unlike  classical  planning,  HTN  planning 
is  in  general  undecidablc,  so  no  sound  and  complete  algorithm  will  terminate  on 
all  problems.  The  objective  of  this  thesis  is  to  study  the  underlying  computational 
complexities  of  using  search  techniques  to  solve  HTN  planning  problems.  The  work 
includes: 

•  Developing  heuristic  search  algorithms  that  terminate  on  large  subsets  of  HTN 
problems,  including  the  syntactically-identihable  sets  of  HTN  problems  which 
are  decidable  [Erol  et  ah,  1996] 

•  Classifying  what  subsets  of  HTN  problems  these  algorithms  terminate  on, 
describing  the  expressivity  of  those  sets  and  providing  run-time  bounds  of  the 
algorithms  on  those  sets. 
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•  Establishing  the  complexity  of  delete-free  HTN  planning,  which  will  have  sig¬ 
nificant  impact  on  the  development  of  domain  independent  heuristics  for  HTN 
planning. 

•  Providing  a  proof  of  concept,  showing  that  domain  independent  heuristics 
and  an  understanding  of  the  search  space  can  lead  to  more  effective  search 
techniques. 

This  work  is  organized  into  three  chapters,  described  below. 

Chapter  2  presents  four  algorithms  for  HTN  planning.  Two  of  these  algo¬ 
rithms  can  be  directly  adapted  for  efficient  heuristic  search.  The  other  two  rely 
on  an  AND/OR  search  which  is  less  amenable  to  many  heuristic  search  techniques. 
Each  algorithm  is  associated  with  a  class  of  problems,  identifiable  syntactically  in 
polynomial  time,  on  which  the  algorithm  is  guaranteed  to  terminate.  The  chapter 
provides  upper  and  lower  complexity  bounds  on  these  classes,  including  new  results 
for  previously  known  decidable  classes. 

Chapter  3  examines  the  complexities  of  delete-free  HTN  planning.  In  classical 
planning,  the  ability  to  find  a  plan  in  domains  with  positive  preconditions  and 
no  negative  effects  in  polynomial  time  is  fundamental  to  most  classical  planning 
heuristics.  I  prove  that  almost  all  propositional  delete-free  HTN  planning  is  NP- 
complete.  This  means  that,  unless  P  =  NP,  that  any  polynomial  time  heuristic 
based  on  solving  delete-free  HTN  problems  must  also  relax  the  semantics  of  HTN 
planning. 
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Chapter  4  presents  a  technique  for  translating  a  subset  of  HTN  problems  into 
classical  planning  problems  (in  specific,  PDDL  [Fox  and  Long,  2003]).  This  means 
that,  even  without  a  dedicated  HTN  heuristic,  any  PDDL  planner’s  heuristic  be¬ 
comes  an  HTN  planning  heuristic  when  used  on  the  translated  problem.  Moreover, 
since  most  classical  planners  are  guaranteed  to  terminate,  they  can  be  used  to  prove 
whether  any  translatable  problem  is  solvable  or  not.  My  experiments  show  that 
for  the  subset  of  problems  that  are  translatable,  planning-via-translation  can  be  an 
effective  way  to  find  HTN  plans. 

1.1  Related  Work 

Kutluhan  Erol’s  seminal  work  on  HTN  planning  theory  [Erol  et  ah,  1996] 
formalized  HTN  planning,  as  well  as  provided  several  important  complexity  and 
decidability  results.  The  most  significant  result  in  Erol  et  al.’s  work  is  that  HTN 
planning  is  in  general  undecidable  by  showing  how  to  encode  the  intersection  of  any 
two  context-free  grammars  as  an  HTN  problem.  Erol  shows  a  number  of  syntactic 
restrictions  of  HTN  planning  are  decidable.  Chapter  2  discusses  these  decidable 
fragments  further. 

Geier  and  Bercher  [2011]  show  that  you  can  regain  decidability  by  relaxing 
what  it  means  to  be  a  solution  to  an  HTN  problem.  If  one  allows  arbitrary  op¬ 
erators  to  be  inserted  into  the  final  plan,  then  any  solution  to  a  problem  can  be 
mimicked  by  an  acyclic  decomposition  of  the  initial  task  network  plus  operator  in¬ 
sertion.  This  means  that  a  Task-Insertion  HTN  (TIHTN)  planner  would  only  need 
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to  explore  acyclic  decompositions  of  the  initial  task  (which  are  finite)  and  if  those 
decompositions  can  be  made  executable  through  task  insertion  (which  is  decidable). 
Chapter  3  discusses  task-insertion  HTN  planning  further,  showing  that  hireling  a 
plan  in  T1HTN  problems  with  positive  preconditions  and  effects  can  be  done  in 
polynomial  time. 

SHOP2  [Nau  et  ah,  2003]  is  the  most  well  known  HTN  planning  implementa¬ 
tion,  known  for  both  efficiency  and  flexibility.  SHOP2  plans  for  tasks  in  the  order 
they  are  to  be  executed,  and  can  be  configured  to  use  either  depth-  or  breadth-first 
search.  Normally  run  in  depth-first  search  mode,  SHOP2  requires  that  the  author 
order  the  methods  such  that  every  search  path  terminates  or  ends  in  a  solution.  HT- 
NPBP  [Sohrabi  et  ah,  2009]  is  a  preference-based  HTN  planner  based  on  SHOP2, 
which  performs  a  best-first  search  over  state-based  preferences,  and  it  has  the  same 
method-ordering  requirements  that  SHOP2  has. 

Elkawkagy  et  ah  [2012]  developed  the  Landmark-Aware  heuristic  HTN  plan¬ 
ner.  The  landmark-aware  heuristic  analyzes  what  tasks  must  occur  in  any  solution 
to  that  plan,  then  orders  the  plans  by  which  one  has  the  fewest  required  tasks, 
and  thus  requires  the  least  effort  to  turn  into  a  solution.  The  planner  itself  is  a 
refinement-based  HTN  planner  which  maintains  a  set  of  partially  refined  plans  or¬ 
dered  by  its  heuristic.  At  each  iteration  the  planner  removes  the  most  promising 
plan  it  has  found,  produces  a  set  of  possible  refinements  of  the  plan. 

Bercher  and  Biundo  [2012]  developed  a  heuristic  for  preference  based  planning 
in  a  hybrid  HTN  and  partially-ordered  causal  link  framework.  Given  a  task  network, 
they  take  the  subset  consisting  of  just  the  primitive  operators,  and  translate  that 
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into  a  classical  planning  problem.  They  then  run  a  Relaxed  Graphplan  variant  to 
estimate  the  best  quality  solution  to  this  problem. 

1.2  HTN  Planning 

In  this  section,  we  present  a  propositional  HTN  planning  formalism,  using  the 
notation  presented  in  [Geier  and  Bercher,  2011]. 

It  will  be  important  for  us  to  have  a  notation  for  the  restriction  of  a  function 
or  relation  to  some  subset  of  its  domain.  For  this,  we  will  use  a  bar  notation  that 
is  defined  as  follows.  For  a  binary  relation  R  C  A  x  A,  the  restriction  of  R  to  any 
X  C  A  is 

R\x  =  { ipi t P2)  e  R  |  pi,p2  e  X}. 

Similarly,  for  a  function  /  :  P  — y  Q,  the  restriction  of  /  to  any  X  C  P  is 

f\x  =  { f(p )  =  q  |  P&X}. 

Hierarchical  Task  Network  (HTN)  planning  is  the  problem  of  decomposing 
an  initial  task  into  a  sequence  of  executable  steps.  A  task  is  an  activity  to  be 
accomplished  labeled  with  a  task  name,  which  is  a  proposition  symbol,  and  will  be 
either  primitive,  corresponding  to  a  concrete  action,  or  compound,  representing  an 
abstract  activity. 

Given  a  set  of  task  names  X,  a  task  network  is  a  tuple  tn  =  (T,  — <,  ok)  such 

that: 

•  T  is  a  finite  nonempty  set  of  task  symbols. 
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•  -<  is  a  partial  order  over  T. 


•  a  :  T  — >  X  is  a  mapping  from  the  task  symbols  to  a  finite  set  of  task  names. 

The  task  symbols  function  as  place  holders  for  task  names,  allowing  multiple  in¬ 
stances  of  a  task  name  to  exist  in  a  task  network  [Erol  et  ah,  1996].  We  say  a 
task  network  (T,  -<,a)  is  equivalent  to  another  task  network  (T',  a')  if  there  is 

an  isomorphism  0  :  T  — >  V  such  that  Vto,ti  E  T  \  (to  -<  t\)  (0(to)  0(0i))  and 

Vt  G  T  |  a  (t0)  =  a'  (0  (to))-  We  refer  to  the  set  of  all  task  networks  over  a  set  of 
task  names  A"  as  TNx- 

An  HTN  domain  is  a  tuple  (L,  C,  O,  M),  where  L  is  a  function- free  first  order 
language,  C  is  a  finite  set  of  compound  task  names,  O  is  a  finite  set  of  primitive  task 
names,  and  M  C  C  x  TNquo  is  a  set  of  methods  over  C  and  O.  The  proposition 
symbols  in  C ,  O  and  L  are  pairwise  disjoint. 

Each  primitive  task  name  o  G  O  is  associated  with  a  planning  operator ,  which 
is  the  triple  (prec(o),  add(o ),  del(o))  where  prec(o)  is  a  propositional  formula  over  L 
and  add(o)  and  del(o)  are  disjoint  subsets  of  literals  from  L.  The  combined  set  of 
free  variables  of  prec(o),  add(o),  and  del(o)  are  the  parameters  of  o. 

Note  that  the  semantic  models  for  the  operators  in  O  forms  an  implicit  state 
transition  function  for  the  planning  domain: 

7  :  2l  x  O  2l, 

where: 

•  A  state  is  any  subset  of  L.  The  finite  set  of  states  in  a  planning  domain  is 
denoted  as  2L  in  the  above  definition  of  7; 
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•  j(s,  o)  is  defined  iff  s  \=  prec(o );  and 

•  7 (s,  o)  —  (s  \  del(o ))  U  add(o ). 

We  will  refer  to  the  particular  7  defined  by  a  domain  D  as  7 d,  or  just  7  if  it  is  clear 
from  the  context. 

We  call  a  task  network  primitive  if  a(t)  G  O  for  every  t  G  T.  Otherwise,  the 
task  network  is  non-primitive. 

We  can  decompose  a  non-primitive  task  network  tn  1  =  (Tj,  -<7,07)  if  there  is 
a  non-primitive  task  t  G  Tj  such  that  a(t)  G  (7  and  has  a  corresponding  method 
m  =  (a(t),  (Tm,  -<m,am ))  G  M.  More  formally,  we  define  the  notion  of  task  decom¬ 
position  as  follows.  Assume  without  loss  of  generality  that  7j  0  Tm  —  0.  Then  the 
decomposition  of  tn\  by  m  into  a  task  network  tn 2  (written  tn \  tn-i)  is  given 

by: 

T[  -  Ti  \  {«}  ; 

T2  :=  U  Tm; 

~ <2  :=-<i  \t[ 

U  -<m 

U  {(ti,  ^2)  GT(x  Tm  |  (ti,  t)  G-<i} 

U  { (O,  £1)  G  Tm  x  T(  j  (£,  t\ )  G-<i}  ; 

«2  :=  U  am; 
tn 2  :=  (T2,  -<2,  0:2)  • 

If  there  is  a  finite  sequence  of  task  decompositions  from  tn \  —tD  tn2  — )•£>  ...  —tp  tnm 
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then  we  write  tn i  ~^*D  tnn. 

An  HTN  planning  problem  is  a  tuple  (D,  s0,tn0),  where  D  =  (L,C,0,  M)  is 
an  HTN  domain,  So  G  2L  is  a  state  in  D ,  and  tno  =  ({^o},  0,  {(hn^o)})  is  the  initial 
task  network  containing  a  single  task  xq. 

A  task  network  tn  is  executable  in  a  state  So  for  domain  D  if  tn  is  primitive  and 
there  exists  some  total  ordering  (consistent  with  -<  in  tn)  over  the  tasks  t\,  ...  ,tn 
and  the  sequence  of  states  si,...,sn  that  arise  from  applying  the  primitive  tasks 
(i.e.,  actions)  ti, . . .  ,tn  in  that  order  in  the  initial  state  So:  i.e., 

^i=0...n—  ^(^*+l))  ®i+l 

We  say  that  tn*  is  an  HTN  solution  to  a  planning  problem  P  =  (. D ,  s0,  tn0 )  if 
tn*  is  executable  in  so  and  tn  —t*D  tn*.  In  this  paper,  we  are  concerned  with  two 
HTN  decision  problems:  plan- existence,  for  whether  a  problem  has  any  solution,  and 
k -length-plan- existence,  for  whether  a  problem  has  a  solution  of  k  or  fewer  operators. 

1.3  Classical  Planning 

Classical  planning  is  the  problem  of  finding  a  sequence  of  operators  that  takes 
one  from  the  starting  state  to  a  goal  state.  Unlike  HTN  planning,  there  are  no 
tasks,  and  operators  can  be  applied  to  the  state  whenever  the  state  supports  their 
preconditions. 

Formally,  a  classical  planning  problem  is  the  tuple  (L,  O,  s,  g),  where  L  is  a  set 
of  propositions,  O  is  a  set  of  planning  operators  defined  as  above,  s  C  Lis  the  initial 
state,  and  g  is  the  goal,  specified  as  a  propositional  formula.  L  and  O  implicitly 


form  a  state  transition  function  7  defined  identically  to  the  7  in  the  previous  section. 

A  planning  problem  is  solvable  if  either  s  |=  g  or  if  there  is  some  operator  o 
such  that  7 (s,  o )  =  s'  and  s'  is  solvable.  We  refer  to  this  specific  formulation  of 
classical  planning  as  STRIPS  [Fikes  and  Nilsson,  1972],  This  is  an  implicit  graph 
search  problem.  Bylander  [1994]  and  Erol  et  al.  [1995]  show  that  STRIPS  is  PS  PACE- 
complete. 

Both  plan-existence  and  fc- length-plan-existence  are  the  same  for  both  HTN 
planning  and  STRIPS. 
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Chapter  2:  HTN  Problem  Spaces 


Unlike  HTN  planning,  classical  planning  is  decidable  [Erol  et  at.,  1995].  Thus 
it  is  possible  to  guarantee  the  termination  of  many  classical  planners,  through  the 
use  of  loop-checking  tests  to  prevent  the  planner  from  generating  infinite  cyclic  paths 
through  a  finite  search  space. 

In  contrast,  HTN  planning  is  only  semi-decidable  [Erol  et  ah,  1996],  and  ev¬ 
ery  sound  and  complete  HTN  planner  has  an  infinite  set  of  problems  on  which  it 
will  never  return.  Although  some  syntactic  restrictions  of  HTN  planning  are  fully 
decidable,  efficient  loop-detection  tests  have  not  yet  been  developed,  and  there  is  a 
gulf  between  the  classes  of  HTN  problems  that  are  known  to  be  decidable,  and  the 
classes  of  problems  on  which  current  HTN  planners  can  guarantee  termination. 

Part  of  the  reason  for  this  gulf  is  that  much  less  is  understood  about  the  search 
spaces  of  HTN  planners  than  those  of  classical  planners.  Different  kinds  of  HTN 
planners  have  different  ways  of  combining  plan  generation  with  task  decomposition — 
and  the  structure  of  the  problem  space  can  vary  greatly  depending  on  how  those 
things  are  done. 

The  following  sections  show  how  to  search  for  solutions  to  HTN  problems  while 
retaining  broad  decidability  guarantees: 
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1.  I  characterize  HTN  planning  as  a  search  of  a  problem  space  in  which  each  node 
is  an  HTN  planning  problem.  I  present  four  different  classifications  of  HTN 
problem  spaces: 

•  Decomposition  Space  (DS),  which  UMCP  [Erol  et  ah,  1994]  and  the 
Landmark-Aware  HTN  Planner  [Elkawkagy  et  ah,  2012]  search; 

•  Progression  Space  (PS),  which  SHOP  and  SHOP2  [Nan  et  ah,  1999,  2003] 
and  HTNPBP  [Sohrabi  et  ah,  2009]  search; 

•  Total  Order  Partition  decomposition  and  progression  spaces  (TODS  and 
TOPS),  two  new  kinds  of  problem  spaces  for  which  there  are  not  yet  any 
existing  planners. 

2.  Each  section  provides  sufficient  (and  in  one  case,  necessary)  conditions  to 
guarantee  that  each  kind  of  problem  space  will  be  finite.  These  conditions  can 
be  evaluated  up-front  to  see  if  an  HTN  planning  problem  has  a  finite  problem 
space.  For  each  kind  of  problem  space,  1  show  that  there  are  no  broader 
conditions  for  finiteness  that  look  only  at  possible  decomposition  of  tasks. 

3.  For  each  of  the  finiteness  conditions,  I  provide  upper  and  lower  complexity 
bounds,  showing  both  the  expressivity  of  problems  in  those  classes,  and  refin¬ 
ing  current  known  complexities. 

4.  1  describe  simple  loop-detection  tests  that  can  be  added  to  HTN  planners 

that  will  guarantee  termination  when  the  problem  space  is  finite.1  These  tests 

1  State-space  classical  planners  often  use  a  loop-detection  test  of  the  form  “have  we  seen  this 
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appear  to  be  applicable  to  a  wide  variety  of  HTN  planning  problems,  and 
their  use  will  allow  HTN  planners  to  terminate  in  cases  where  they  would  not 
otherwise  do  so. 

5.  The  HTN-to-PDDL  translation  algorithm  described  in  Chapter  4  requires  a 
user-specified  upper  bound  on  the  HTN  recursion  depth,  and  the  translation 
is  correct  only  when  this  bound  is  sufficiently  high.  By  characterizing  the 
translation  as  a  mapping  from  HTN  progression  spaces  into  classical  state 
spaces,  one  can  compute  a  correct  bound  automatically  whenever  the  finiteness 
conditions  in  Item  2  are  satisfied. 

6.  I  show  that  TODS  and  TOPS  are  finite  for  strictly  broader  classes  of  prob¬ 
lems  than  DS  and  PS,  and  I  provide  new  sound-and-complcte  HTN  planning 
algorithms  for  TODS  and  TOPS.  The  algorithms  are  guaranteed  to  terminate 
whenever  the  problem  space  is  finite. 

2.1  Decomposition  Problem  Spaces 

The  definition  of  HTN  solvability  leads  to  a  natural  definition  of  a  problem 
space  as  a  directed  graph,  where  nodes  are  task  networks,  and  edges  are  decompo¬ 
sitions  from  one  task  network  to  another.  The  initial  task  network  of  a  problem 
forms  the  root  of  the  graph,  and  it  is  solvable  if  and  only  if  there  is  a  path  in  the 
graph  from  it  to  a  primitive  executable  task  network. 

state  before?”  In  contrast,  the  loop-detection  tests  here  are  basically  “have  we  seen  this  problem 
before?” 
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Formally,  for  an  HTN  domain  D  =  (L,  C,  O,  M),  the  directed  graph  (V,E) 
is  the  decomposition  problem  space  of  an  HTN  problem  ( D,s0,tn0 )  if  and  only  if 
(V,  E)  is  the  minimal  graph  containing  trio  such  that  tn  G  V  and  tn  — >d  tn'  implies 
that  tn'  G  V  and  (tn,  tn')  G  E. 


Input:  D  =  (L,C,0,  M)  -  an  HTN  domain 
Input:  ( so, trio )  -  an  initial  state  and  task  network 
V  Fringe  •(—  {tno}; 
while  Fringe  A  0  do 

Choose  and  remove  some  tn  G  Fringe', 
if  tn  is  primitive  and  (D,so,tn)  is  executable  then 
j  return  tn; 

children  G-  {tn'\tn  — >£>  tn'}; 

Fringe  G-  Fringe  U  ( children  \  V); 

V  <—  V  U  children; 

return  fail; 

Figure  2.1:  DHTN(T),  s0,  trio)  A  simple  decomposition  based  HTN  planner. 


Algorithm  2.1  (DHTN)  shows  a  simple  decomposition  space  HTN  planner. 
DHTN  starts  off  with  the  initial  task  network  tn,  and  maintains  a  set  of  known 
decompositions  V  and  a  fringe  of  unexpanded  decomposition.2  At  every  iteration, 
it  chooses  some  task  network  in  its  fringe.  If  tn  is  primitive  and  executable,  DHTN 
returns  tn.  Otherwise,  it  removes  tn  from  the  fringe  and  adds  tn' s  immediate 
decompositions.  If  at  any  time  the  fringe  is  exhausted,  DHTN  returns  failure. 

The  computation  Fringe  <—  Fringe  U  ( children  \  V )  guarantees  that  DHTN 
will  never  add  a  previously  visited  task  network  to  the  fringe.  This  is  a  simple 
loop-detection  test  that  can  be  added  to  other  HTN  planning  algorithms. 

Given  an  HTN  problem  P,  DHTN  is  sound,  complete,  and  terminating  for 
2Fringe  and  V  functionally  correspond  to  the  open  and  closed  sets  of  A*,  respectively. 
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any  problem  for  which  the  decomposition  problem  space  is  finite.  However,  if  the 
problem  space  is  infinite,  DHTN’s  completeness  depends  on  how  it  picks  elements 
out  of  the  fringe  (with  some  acceptable  choices  being  first-in-first-out  or  taking  the 
network  with  the  fewest  tasks).  If  the  problem  has  no  solution  and  the  problem 
space  is  infinite,  then  DHTN  will  not  terminate  no  matter  how  elements  are  chosen 
from  the  fringe. 

Relation  to  other  work. 

The  HTN  decomposition  problem  space  formalizes  the  spaces  searched  by 
existing  planners  such  as  UMCP  [Erol  et  ah,  1994]  and  Elkawkagy’s  landmark- aware 
HTN  planner.  Unlike  DHTN,  neither  UMCP  nor  Elkawkagy’s  planner  check  to  see 
if  they  have  already  expanded  a  problem  before,  so  a  finite  decomposition  space  is 
not  enough  to  ensure  their  termination. 

2.1.1  Decidability  under  decomposition 

For  an  HTN  domain  D  =  (L,  C,  O,  M),  a  non-recursive  task  name  is  one  which 
has  a  finite  k -lev el-mapping,  k  is  a  partial  function  from  C  U  O  — >  Z+  defined  via 
induction: 

•  For  all  o  £  O,  k(o)  =  0. 

•  For  c  £  C,  if  there  is  a  finite  number  n  £  Z+  such  that  n  is  the  greatest  h- level 
of  any  subtask  of  c  in  M,  then  k(c)  —  n  +  1. 
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If  a  task  in  a  task  network  has  a  /c-lcvel,  any  decomposition  of  that  task  replaces 
it  with  a  set  of  tasks  which  have  a  lower  £;-level.  Since  we  can  only  repeat  this  a 
finite  number  of  times,  this  leads  to  one  of  the  first  results  of  [Erol  et  ah,  1996]: 

Theorem  2.1  Let  P  =  (. D,so,tno )  be  an  HTN  -planning  problem  with  every  task 
in  tn0  has  a  finite  k-level  mapping.  Then  the  decomposition  problem  space  for  P  is 
finite. 


So  the  decomposition  problem  space  is  finite  for  non-recursive  problems,  but 
limited  recursion  is  fine  as  long  as  it  does  not  increase  the  size  of  the  task  network. 
It  turns  out  we  can  syntactically  identify  every  problem  for  which  the  problem  space 
is  finite.  To  this  end,  we  say  that  a  task  network  tn  is  <i-stratifiable  if  there  exists 
a  total  preorder  <1  on  the  reachable  task  names  of  tn  such  that  if  c  is  a  reachable 
task  name  of  tn  and  (c,  (T,  — <,  a;))  is  a  method,  then  either: 

•  If  the  task  network  contains  only  one  task  ( T  =  {£}),  then  the  task  name  must 
not  be  on  a  higher  stratum  (aft)  <1  c) 

•  Otherwise,  all  task  names  must  be  on  a  lower  stratum  ( \/teTa(t )  <x  c). 


Example  2.2  Let  P  =  ( D ,  so,tno)  where  trio  contains  the  single  task  r, 


D  =  ( L ,  C,  O,  M),  C  =  {r,  s},  O  =  {a,  b},  and: 


(r,  ({^1}  ,  0,  a(xi)  =  s )) 


M  =  { 


(r,  ({xi}  ,0,a(xi)  =  a)) 
(s,  ({xi},0,a(a;i)  =  r)) 


(s,  ({xh  x2j  ,  0,  (aOi)  =  b,  a(x  2)  =  6})) 
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Then  there  is  a  two  level  <±- stratification,  where  since  r  and  s  decompose  to 
one  another  they  must  be  on  the  same  level,  and  a  and  b  can  be  on  the  lower  level 
together,  since  they  are  primitive. 

Theorem  2.3  Let  P(D,  s0,tn0)  be  an  HTN  planning  problem.  Then  the  decompo¬ 
sition  problem  space  for  P  is  finite  if  and  only  iftn0  is  <i-stratifiable. 

Proof.  (4=)  If  there  exists  a  <i-strati£cation,  then  every  decomposition  either 
produces  a  task  network  of  the  same  size,  or  replaces  a  task  with  a  set  of  tasks 
whose  task  names  are  in  a  lower  level.  Since  the  stratification  is  finite,  the  number 
of  times  repeated  decomposition  can  increase  the  size  of  the  task  network  is  also 
finite. 

(=b)  If  there  is  no  <i-stratikcation,  then  the  transitive  <1  constraints  are 
inconsistent.  This  means  there  exists  a  task  name  c  and  method  m  =  ( c,tnm )  with 
more  than  one  subtask  such  that  c  <1  ...  <1  c,  and  so  c  is  a  reachable  subtask 
of  tnm.  Since  c  is  a  reachable  subtask  of  brio ,  we  can  decompose  tno  into  a  task 
network  containing  c,  then  repeatedly  use  the  decomposition  chain  going  through 
m  to  create  a  task  network  of  arbitrary  size.  Since  any  finite  problem  space  has 
a  bound  on  the  size  of  task  networks  appearing  in  it,  the  problem  space  for  P  is 
infinite.  □ 

Given  an  HTN  domain  D ,  we  can  find  a  <i-strati£cation  of  a  task  network 
tn  =  ( T,~<,a )  in  time  polynomial  in  D  by  performing  a  topological  sort  of  the 
task  names  given  the  constraints.  If  this  procedure  fails,  the  decomposition  space  is 
infinite.  If  it  succeeds,  it  returns  a  stratification.  If  the  height  of  the  stratification  is 
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h  and  the  maximum  number  of  tasks  in  a  method  is  b,  then  the  largest  task  network 
in  the  decomposition  space  of  an  HTN  problem  ( D,s,tn )  is  bounded  by  \T\  ■  bh. 

2.1.2  Complexity  of  <i-stratifiable  problems 

It  is  not  immediately  clear  how  expressive  <i-stratifiable  HTN  planning  is, 
nor  how  long  we  can  expect  a  decomposition  based  planner  to  take  to  solve  those 
problems.  In  this  section,  we  provide  upper  and  lower  complexity  bounds  for  <1- 
stratihable  problems.  In  particular,  <!-stratihable  HTN  planning  is  at  least  as 
expressive  as  STRIPS-style  planning. 

Theorem  2.4  Deciding  plan- existence  and  k -length-plan- existence  <i-stratifiable 
problems  is  in  NEXPTIME  (non- deterministic  exponential  time). 

Proof.  First,  show  that  we  can  compile  any  <i-stratihablc  domain  to  a  domain 
with  a  Avlevel  mapping  in  polynomial  time.  Let  1  be  a  task  such  that  t  <i  t\  <i 
*2  <1  •  •  •  <1  tn.  Then  by  adding  a  method  (t,  tn)  to  the  domain  for  every  method 
(ti,tn)  and  then  removing  every  method  ( t,tn ')  where  tn'  contains  one  of  the  £*,  we 
transform  the  domain  to  one  with  the  same  set  of  solutions,  but  now  t  can  exist  on  a 
separate  strata  then  H, . . .  ,tn.  If  we  repeat  this  process,  we  end  up  with  a  problem 
that  has  the  same  set  of  solutions  as  the  original,  but  is  now  has  a  h-level  mapping 
with  no  recursion. 

With  no  recursion,  we  can  only  decompose  the  task  network  at  most  mk  times, 
where  m  is  the  size  of  the  largest  method  and  k  is  the  minimum  /e-level  mapping. 
So  searching  for  a  solution  to  <i-stratifiable  problems  is  in  NEXPTIME.  □ 
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We  can  view  the  NEXPTIME  result  as  the  gap  between  a  blind  search  of  the 
progression  space  and  using  a  perfect  heuristic  to  guide  the  search.  If  DHTN  is  given 
oracle  access  to  a  perfect  heuristic  (one  that  estimates  the  exact  size  of  the  minimum 
solution  to  any  problem),  then  always  selecting  the  problem  from  its  fringe  with  the 
minimum  heuristic  value  will  lead  DHTN  to  terminate  in  exponential  time  on  <r 
stratifiable  problems.  On  the  other  hand,  if  it  selects  poorly  and  expands  every  node 
in  the  decomposition  space  before  terminating,  it  may  take  double-exponential  time. 

Erol  et  al.  [1996]  shows  that  HTN  planning  with  /c-level  problems  is  decidable, 
but  does  not  give  a  lower  complexity  bound.  By  mapping  STRIPS  into  a  <1- 
stratifiable  problem,  we  can  give  a  PS  PACE  lower  bound: 

Theorem  2.5  Deciding  plan- existence  and  k -length-plan- existence  for  <i-stratifiable 
problems  is  PSPACE -hard. 

Proof.  Let  P  =  ( L,0,s,g )  be  a  STRIPS  problem.  Here  we  encode  P  as  a  <1- 
stratifiable  HTN  problem  with  the  same  set  of  solutions.  We  will  take  advantage  of 
the  fact  the  minimum  length  solution  to  any  classical  plan  does  not  visit  the  same, 
and  so  must  be  of  length  2^  —  1  or  less. 

Let  O'  be  a  set  of  primitive  tasks  containing  tasks  for  every  operator  in  O, 

plus: 

•  A  no-op  task,  whose  operator  has  no  preconditions  or  effects 

•  A  task  og,  whose  operator  has  a  precondition  of  g  and  no  effects. 

Let  C  contain  the  set  of  task  names  a \ . ,  am,  and  M  contain: 
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•  (ai,  tnno-op),  where  £nno_op  contains  only  the  no-op  task. 

•  A  method  ( ai,trii )  for  each  primitive  task  ot  e  O'  \  {o9},  where  trii  contains 

the  task  ty. 

•  A  method  ( aj,trij )  for  2 . . .  \L\,  where  trij  contains  aj_ i  followed  by  itself. 

Let  the  initial  task  network  tn  contain  the  tasks  cl\l\  followed  by  g'.  Then 
the  <!-stratifiable  problem  Pl  =  ((L,C,  0',M)  ,s,tn )  can  be  decomposed  into  any 
sequence  of  2^LI  —  1  no-ops  and  operators  from  O,  followed  the  operator  og  which 
tests  for  the  goal  condition  g.  Thus  any  to  P  implies  a  solution  to  P',  and  vice-versa. 

Since  STRIPS  is  PS  PACE-complete  [Bylander,  1994],  ^-stratifiable  HTN  plan¬ 
ning  is  PS  PACE- hard.  □ 

This  gives  a  loose  bound  on  the  complexity  of  planning  for  /ei-stratihable 
problems.  The  NEXPTIME  upper  bound  is  based  on  the  exponential  relationship 
between  the  height  of  a  <i-stratihable  problem’s  minimum  stratification  and  the 
maximum  size  of  a  task  network  reachable  under  decomposition.  However,  many 
HTN  planning  problems  have  a  fixed  small  stratification  height,  and  problem  in¬ 
stances  only  vary  by  the  size  of  the  state  space  and  the  number  of  tasks  in  the  initial 
task  network.  Tighter  bounds  can  be  found  by  parameterizing  the  class  based  on 
the  height  of  a  problem’s  smallest  stratification.  For  a  constant  c,  we  say  a  problem 
is  <5-strati£ablc  if  there  exists  a  <i-strati£cation  with  a  height  of  c  or  less.  This 
matches  with  common  practice  in  HTN  planning,  since  most  domains  have  a  fixed 
(and  small)  stratification  (see  Section  2.4). 
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Then  the  largest  task  network  reachable  under  decomposition  is  bounded  by 
a  polynomial  of  degree  c+  1.  Later  on,  Section  3.3  shows  that  finding  an  executable 
decomposition  of  a  problem  of  a  fixed  size  is  in  NP.  Meanwhile,  Erol  et  al.  [1996] 
shows  that  merely  ordering  the  leaves  of  this  tree  is  NP-hard,  leading  to  a  tight  and 
much  reduced  bound  for  ^-problems: 

Theorem  2.6  For  fixed,  c,  then  deciding  plan- existence  or  k -length- plan- existence 
for  <l-stratifiable  HTN  problems  is  UP -complete. 

2.2  Progression  Problem  Spaces 

Decomposition  spaces  are  not  the  only  way  to  solve  HTN  planning  problems. 
Let  P  =  ( D,s,tn )  be  an  HTN  planning  problem,  where  D  =  (L,C,0,  M)  and 
tn  =  (T,  -<,a).  If  there  exists  a  task  t  G  T  with  no  predecessors  (one  such  that 
Vt'erf  ~f\  t )  and  its  operator  aft)  6  O  is  applicable  in  s,  then  we  can  progress  the 
problem  from  ( D,s,tn )  to  the  problem  P'  =  (D,ryD(s,a(t))  ,tn\{t})  (where  the 
notation  tn  \  {t}  simply  means  removing  any  occurrence  of  t  from  T,  -<,  and  a).  If 
there  exists  an  unconstrained  task  t  G  T  which  is  non- primitive  (aft)  G  C),  then 
any  decomposition  tn  tn'  is  a  valid  progression  of  ( D,s,tn )  to  ( D,s,tn' ).  We 

write  progression  as  P  P'. 

Intuitively,  a  progression  interleaves  a  decomposition  with  imposing  an  exe¬ 
cutable  total  order  over  the  primitive  tasks.  The  following  theorem  establishes  an 
equivalence  between  these  two  paradigms: 

Theorem  2.7  An  HTN  problem  P  is  solvable  if  and  only  if  P  is  executable  or  there 
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exists  a  P'  such  that  P  —tp  P'  and  P'  is  solvable. 


Progression,  then,  also  leads  to  a  natural  definition  of  the  problem  space  as 
a  directed  graph.  The  progression  problem  space  of  an  HTN  problem  (D,  so,tno) 
is  the  minimal  directed  graph  (V,  E)  containing  (so,tno)  such  that  ( s,tn )  G  V  and 
(. s,tn )  — >p  (s' ,  in')  implies  that  (s' ,  in')  G  V  and  ((s,tn),(s' ,tn'))  G  E. 


Input:  D  =  (L,C,0,  M)  -  an  ffTN  domain 
Input:  ( so,tno )  -  an  initial  state  and  task  network 
V  <—  Fringe  •*—  {(so,tno)};  E  G-  0; 
while  Fringe  7^  0  do 

Choose  some  (s,tn)  G  Fringe ; 
if  tn  is  primitive  and  (. D,s,tn )  is  executable  then 
J  return  path  in  (V,  E)  from  (so,tno)  to  ( s,tn ); 

children  <—  {( s\tn')\(s,tn )  — >p  (s',tn')}; 

Fringe  <—  ( Fringe  \  {(s,tn)})  U  ( children  \  V); 

V  <—  P  U  children ; 

E  E  U  {((s,  tn),  (-s',  tn'))  |  (s',  tn')  G  children }; 

return  fail; 

Figure  2.2:  PHTN(id,  so,  tn0)  A  simple  progression  based  HTN  planner. 


PHTN  (Algorithm  2.2)  is  a  simple  progression-based  HTN  planner.  PHTN 
maintains  a  directed  graph  of  HTN  problems  reachable  from  the  initial  problem  and 
expands  problems  from  the  leaves  of  this  graph.  When  it  encounters  a  primitive  ex¬ 
ecutable  problem,  it  returns  the  entire  sequence  of  progressions  from  initial  problem 
to  its  solution. 

PHTN  is  sound,  complete,  and  terminating  for  any  HTN  problem  which  has  a 
finite  progression  problem  space.  The  computation  Fringe  ( Fringe  \  {(s,  tn)}) U 
(< children  \  V)  guarantees  that  PHTN  will  not  add  previously  visited  task  networks 
to  the  fringe.  This  loop-detection  test  can  be  added  to  other  HTN  planning  algo¬ 
rithms.  But  as  with  DHTN,  if  the  problem  space  is  infinite  and  there  is  no  solution, 
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PHTN  will  never  return. 


Relation  to  other  work.  Progression  problem  spaces  provide  an  implicit  for¬ 
malization  of  the  problem  space  behind  several  existing  HTN  planning  works  such 
as  SHOP2  [Nan  et  al.,  2003],  HTNPBP  [Sohrabi  et  al.,  2009]  and  the  HTN-PDDL 
translation  in  the  previous  section.  As  with  decomposition-based  planners,  finite¬ 
ness  does  not  guarantee  termination;  e.g.,  neither  SHOP2  nor  HTNPBP  check  if 
they  have  already  expanded  a  problem. 

2.2.1  Decidability  under  progression 

Given  an  HTN  domain  D,  suppose  that  the  task  network  for  every  method  in 
a  domain  consisted  of  a  set  of  primitive  tasks  and  at  most  one  non-primitive  task 
which  is  constrained  to  come  after  them.  Erol  et  al.  [1996]  call  this  a  regular  domain 
and  prove  a  decidability  result  for  regular  HTN  domains.  We  adapt  their  result  to 
our  progression  problem  spaces  as  follows: 

Theorem  2.8  Given  a  regular  HTN  domain  D ,  any  HTN  problem  P  =  ( D ,  s0,  trio) 
has  a  finite  progression  problem  space. 

The  proof  follows  Erol  et  al.’s  results.  Intuitively,  given  any  problem  P  =  ( D ,  s0,  trio) 
with  a  regular  domain,  no  progression  can  increase  the  number  of  non-primitive  tasks 
in  a  task  network.  Furthermore,  any  primitive  task  introduced  along  with  a  non¬ 
primitive  task  must  be  progressed  out  of  a  task  network  before  the  non-primitive 
task  can  be  decomposed.  Thus,  this  bounds  the  size  of  the  task  networks  in  the 
progression  problem  space  of  P. 
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We  can  extend  this  class  of  decidable  problems  using  the  same  stratification 
technique  we  used  for  decomposition.  A  task  network  tn  is  <r-stratiftable  if  there 
exists  a  total  preorder  <r  on  the  reachable  task  names  of  tn  such  that  for  every 
method  (c,  (T,  -<,«))  with  a  reachable  task  name  c: 

•  If  there  is  a  task  tr  G  T  such  that  all  other  tasks  are  predecessors  (\/teTt^trt  -< 
tr),  then  a(tr)  <r  c.  We  call  tr  the  last  task  of  (T,  -<,  a). 

•  For  all  non-last  tasks  t  G  T,  a(t )  <r  c. 

If  an  HTN  planning  problem  P’s  task  network  is  <r-stratihable,  then  any 
progression  P  replaces  a  task  with  at  most  one  task  of  the  same  level,  with  the  rest 
occurring  at  a  lower  level.  Since  the  lower  level  tasks  are  constrained  to  come  before 
this  task,  they  must  be  progressed  out  of  the  task  network  before  this  task  can  be 
decomposed.  Since  the  stratification  is  finite,  this  gives  us  a  bound  on  the  maximum 
size  of  the  network,  and  produces  our  next  finiteness  result: 

Theorem  2.9  Given  an  HTN  problem  P  =  (D,  so,tno) ,  if  tn0  is  <r-stratifiable, 
then  the  progression  problem  space  of  P  is  finite. 

What  happens  to  the  problem  space  if  there  is  no  stratification?  Unlike  with 
decomposition,  now  both  the  structure  of  the  transition  function  and  the  set  of 
methods  can  affect  which  problems  are  in  the  problem  space.  This  limits  our  result 
on  when  the  problem  space  must  be  infinite: 

Theorem  2.10  Given  an  HTN  problem  P  =  (D,s0,tno)  su°h  that  every  problem 
in  the  domain  is  solvable  and  there  is  no  <r-stratification  of  the  reachable  subtasks 
of  tn0,  then  the  progression  problem  space  of  P  is  infinite. 
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Proof.  Given  that  there  is  no  < r-str  at  ificat  ion ,  the  <r-constraints  must  be  incon¬ 
sistent,  meaning  there  must  be  two  reachable  task  names  (not  necessarily  distinct) 
such  that  b  <r  c  and  c  <r  b.  So  there  is  a  method  m  =  (c,  (T,  -<,  a)),  and  two 
tasks  ti,t2  €  T  such  that  t\  -/(  t2  and  a(t2)  =  b.  Since  c  is  a  reachable  task  from 
trio  and  every  problem  in  P  is  solvable,  there  is  a  series  of  progressions  such  that  c 
progresses  to  a  task  network  containing  itself.  Since  we  did  not  need  to  progress  t\ 
out  of  trio  in  order  to  expand  t2,  we  can  use  this  loop  to  create  a  task  network  of 
arbitrary  size.  Thus  the  progression  problem  space  of  P  is  infinite.  □ 

Since  <r-strati£able  is  a  strict  broadening  of  the  <i-stratifiability  definition, 
if  a  task  network  is  <i-strati£able,  it  is  also  <r-stratifiable.  Like  <i-strati£cations, 
we  can  find  <r-str  at  ificat  ions  with  a  topological  sort  of  the  reachable  task  names 
of  a  task  network.  For  an  HTN  problem  P  with  the  task  network  tn  =  (T,  -<,  a) 
and  a  stratification  of  tn  of  height  h  and  a  largest  method  size  of  b,  a  task  in  a 
stratum  can  contribute  at  most  1  plus  b  times  the  bound  of  the  strata  below  it  to 
the  maximum  size  of  a  task  network  reachable  under  progression.  This  gives  a  total 
bound  of  \T\  ■  Y2i=o^1  =  \T\  •  for  the  maximum  size  of  a  task  network  in  the 
progression  problem  space  of  P.3 

Relation  to  other  work. 

The  HTN  to  PDDL  translation  algorithm  in  Chapter  4  essentially  maps  the 

progression  problem  space  into  a  classical  domain.  The  algorithm  adds  a  fixed  num- 
3  One  can  do  much  better  than  that  by  directly  inspecting  the  stratification,  but  that  is  beyond 
the  scope  of  this  thesis. 
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ber  of  identifiers,  specified  by  the  user,  to  represent  the  structure  of  the  task  network. 
In  order  for  the  translation  to  be  correct,  there  must  be  more  identifiers  available 
than  there  are  tasks  in  the  largest  task  network  encountered  under  progression.  If 
the  initial  task  network  is  <r-stratifiable,  we  can  use  the  bound  in  the  previous 
paragraph  instead  of  asking  the  user. 

2.2.2  Identifiability  of  finite  progression  spaces 

Since  we  could  identify  the  exact  set  of  problems  which  have  a  finite  decom¬ 
position  space,  there  is  a  natural  question  about  whether  we  can  identify  more 
problems  which  have  a  finite  progression  space. 

Theorem  2.10  says  that  if  the  initial  task  network  is  not  <r-stratihablc,  and 
every  problem  in  the  domain  is  be  solvable,  then  the  progression  space  is  infinite. 
To  say  that  every  problem  in  the  domain  is  solvable  is  equivalent  to  the  following 
two  conditions: 

•  Every  operator  is  applicable  in  every  state  (i.e.,  V<,e2£i0eo>  s  |=  pre(o)). 

•  Every  task  has  a  primitive  decomposition  (transitively,  not  necessarily  an  im¬ 
mediate  decomposition). 

We  can  check  the  second  condition  recursively: 

•  Every  primitive  task  has  a  primitive  decomposition. 

•  Every  task  which  has  a  method  where  all  the  tasks  have  a  primitive  decom¬ 
position  also  has  a  primitive  decomposition. 
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Once  all  tasks  with  a  primitive  decomposition  are  marked,  the  rest  have  no 
primitive  decomposition,  and  so  are  trivially  unsolvable.  Since  any  task  network 
with  a  task  that  can’t  be  decomposed  into  primitive  network  is  unsolvable,  we  can 
preprocess  the  domain,  removing  any  trivially  unsolvable  tasks  and  methods  that 
refer  to  them. 

This  lets  us  identify  every  problem  with  a  finite  progression  space  that  can  be 
identified  without  looking  at  the  preconditions  and  effects  of  the  operators: 

Theorem  2.11  Let  P  be  an  HTN  -planning  problem  and  let  P'  be  the  HTN  planning 
problem  where  all  of  the  trivially  unsolvable  tasks  of  P  are  removed.  If  P'  is  not 
<r-stratifiable,  then  the  following  holds: 

There  does  not  exist  a  function  Q(P )  which,  without  examining  P’s  operators, 
returns  true  if  and  only  if  P  has  a  finite  progression  space. 

Proof.  Let  P  =  ( D,s,tn )  and  P'  =  ( D',s,tn )  be  HTN  problems,  where  D'  is  D 
without  trivially  unsolvable  tasks,  such  that  P  has  a  finite  progression  space  and  P' 
has  no  <r-strati£cation. 

Let  Pjj  and  Pf  be  P  and  P'  where  every  operator’s  precondition  is  set  to  true 
and  every  effect  is  removed.  Then  by  Theorem  2.10,  Pf  has  an  infinite  progression 
space.  Since  adding  methods  and  tasks  cannot  make  a  problem’s  infinite  progression 
space  finite,  Pu  also  has  an  infinite  progression  space. 

So  if  Q(P)  returns  true  and  Q(Pu)  returns  false,  then  Q  must  inspect  the 
operators  of  P  and  Pjj,  since  that  is  their  only  difference.  □ 
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2.2.3  Complexity  of  <r-stratifiable  problems 

We  can  solve  <r-stratifiable  problems  with  a  non-deterministic  search  of  the 
progression  space  that  keeps  track  of  only  the  current  task  network  and  state.  Given 
that  the  height  of  the  stratification  gives  an  exponential  bound  on  the  size  of  task 
networks  encountered  under  progression,  this  provides  an  upper  bound  on  the  com¬ 
plexity  of  planning  for  <r-stratifiablc  problems. 

Theorem  2.12  Deciding  plan- existence  and  k -length-plan- existence  <r- strati fiable 
problems  is  in  EX  PS  PACE. 

As  with  <i-stratifiable  problems,  we  can  parameterize  the  class  of  <r-stratifiable 
problems  based  on  the  stratification  height.  For  a  constant  c,  we  say  a  problem  is 
<£-stratifiable  if  there  exists  a  <r-stratification  with  a  height  of  c  or  less.  The 
class  of  regular  HTN  problems  from  Erol  et  al.  [1996]  is  identical  to  the  class  of 
<3-stratifiable  problems. 

Theorem  2.13  For  fixed  c,  deciding  plan- existence  and  k -length- plan- existence  <cr- 
stratifiable  problems  is  PS  PACE -complete. 

Proof.  Containment.  For  fixed  c,  task  networks  in  the  progression  space  of 
<£-stratifiable  problems  are  bound  in  size  by  a  polynomial  of  degree  c.  Non- 
deterministic  search  then  only  takes  polynomial  space. 

Hardness.  Given  that  for  c  >  0,  the  class  of  <£-stratifiable  problems  includes 
all  regular  HTN  problems,  for  which  plan-existence  is  PS  PACE-complete  [Erol  et  al., 
1996],  □ 
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2.3  Total  Order  Partition  Problem  Spaces 


This  section  describes  two  new  problem  spaces,  based  on  our  formulation  of 
DS  and  PS.  The  new  problem  spaces  will  allow  us  to  define  new  HTN  planning 
algorithms  which  terminate  for  a  broader  class  of  HTN  problems,  as  described  in 
the  subsequent  section. 

2.3.1  Total  Order  Partitions 

Let  P  =  ( D,s0,tn )  be  an  HTN  planning  problem  where  tn  =  (' T,-<,a )  is  its 
task  network.  Then  the  sequence  of  task  networks  ((To,  -<o,  no)  ,  •  •  • ,  (Tfc,  -<k,  oik))  is 
a  total  order  partition  of  tn  if: 

•  Their  union  equals  tn: 

-  T  =  lie  Ti 

-  ■<=  Ito 

-  a  =  [Ji=o  ai 

•  They  are  disjoint  and  tasks  between  networks  are  ordered,  i.e.  for  any  T%  and 
Tj  such  that  i  <  j: 

-  Ti  n  Tj  -  0 

~  VueTitjeTjU  -<  tj 

A  total  order  partition  is  a  serialization  of  the  task  network  into  smaller  prob¬ 


lems,  which  we  can  attempt  to  solve  the  smaller  problems  sequentially  without 


interactions  from  other  tasks: 


Theorem  2.14  Let  P  =  ( D,s0,tn )  and  let  (tn0, . . .  ,tnk )  be  a  total  order  partition 
of  tn.  Then  P  is  solvable  iff  there  exists  a  sequence  of  states  s i, . . .  ,Sfc+i  such  that 
for  all  i  <  k  ( D,Si,tni )  has  a  solution  with  an  ending  state  of  Si+\. 

Proof.  (<£=)  If  the  sequence  of  partitions  is  solvable,  then  each  tni  decompose  into 
some  primitive  task  network  tn'% .  Applying  these  decompositions  to  the  correspond¬ 
ing  tasks  in  tn  gives  you  a  primitive  task  network  tn'  which  will  have  an  ending 
state  of  si+1. 

(=0  Suppose  P  has  a  primitive  executable  decomposition  tn' .  Since  it  is 
executable,  there  is  a  total  order  over  the  tasks  of  tn.  Since  decomposition  preserves 
ordering,  we  can  split  that  sequence  into  a  solution  for  the  partition.  □ 

Given  that  -<  is  a  consistent  partial  order,  there  will  be  a  unique  longest  total 
order  partition  from  which  none  of  the  reduced  problems  can  be  further  reduced.  If 
the  longest  total  order  partition  contains  only  a  single  task  newtork,  we  call  that  a 
trivial  partition. 

TOD  and  TOP  problem  spaces. 

Total  order  partitions  give  us  two  AND/OR  problem  spaces  for  HTN  planning, 
one  defined  over  decomposition  and  one  over  progression. 

Given  an  HTN  domain  D  =  (. L ,  C,  O,  M ),  the  total-order  decomposition  (TOD) 
problem  space  for  an  HTN  problem  P  =  (D,  so,tno )  is  the  minimal  directed  labeled 
graph  (V,  E)  containing  (s0,tn0)  such  that  for  every  (s,tn)  G  V: 
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•  If  tn  has  only  a  trivial  total  order  partition  and  tn  — >d  tn' ,  then  ( s,tn ')  is  also 
in  V  with  the  edge  ((s,tn),  0,  ( s,tn '))  G  E. 

•  Otherwise,  let  {trii, . . .  ,  tnk}  be  the  longest  total  order  partition  of  tn.  Edges 
will  point  to  reduced  problems,  labeled  with  where  in  the  sequence  the  reduce 
problem  lies.  Then: 

-  (s,tni)  G  V  with  the  edge  (( s,tn ),  1,  (s,tni)). 

-  For  i  <  k,  if  there  exists  an  edge  (( s,tn),i ,  ( s',tni ))  G  E  and  ( s',tui )  has 
a  solution  with  an  ending  state  of  s" ,  then  there  exists  (s",tni+ 1)  G  V 
with  the  edge  (( s,tn),i  +  1,  (s",tni+ 1)). 

For  each  problem  P  =  ( s,tn )  in  the  TOD  problem  space  with  a  non-trivial 
total  order  partition  tni, . . .  ,tnk,  we  label  P’s  outgoing  edges  with  the  integer  cor¬ 
responding  to  its  reduced  task  networked  (edges  corresponding  to  decomposition 
are  given  an  arbitrary  label  of  0).  If  there  is  an  edge  ((s,tn),i,  (si,tni))  G  E 
for  i  >  1,  then  by  the  definition  there  must  also  be  a  state  and  an  edge 
((s,  tn),i  —  1,  (si-i,tni-i))  G  E  such  that  (sj_ i,  Pq_i)  has  a  solution  with  an  ending 
state  of  Si.  So  if  there  is  an  edge  ((s,  tn),  k,  (. s*, ),  tnk))  G  E  such  that  (. D ,  Sk,  tnk)  has 
a  solution  with  an  ending  state  of  sk+ 1,  then  there  must  be  a  chain  of  states  that 
solves  the  partition,  and  so  P  has  a  solution  with  an  ending  state  of 

The  total-order  progression  (TOP)  problem  space  is  defined  similarly  to  the 
TOD  problem  space,  replacing  decomposition  with  progression. 
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2.3.2  Search  in  TOD  and  TOP  spaces 


We  now  describe  two  new  HTN-planning  algorithms,  called  TODHTN  and 
TOPHTN,  which  perform  an  AND/OR  search  over  the  TOD  and  TOP  problem 
spaces,  respectively. 

Algorithm  2.3  shows  a  high-level  description  of  the  TODHTN  procedure. 
TODHTN  maintains  a  set  of  variables  as  PHTN  -  a  directed  but  now  edge-labeled 
graph  (V,  E )  of  HTN  problems,  a  set  of  HTN  problems  (Fringe),  and  a  new  map  A", 
which  maps  HTN  problems  to  a  set  of  known  possible  end  states.  TODHTN  then 
begins  a  two  phase  iterative  process  of  selecting  a  node  from  the  fringe  to  expand, 
then  propagating  the  consequences  through  the  graph. 

Every  iteration  of  TODHTN  selects  a  problem  ( s,tn )  from  the  fringe  and 
examines  its  task  network.  Nothing  is  added  to  the  graph  in  this  phase,  but  instead 
TODHTN  marks  new  edges  and  ending  states  to  add  later  during  the  propagation 
phase.  If  the  task  network  has  a  non-trivial  total  order  partition  (tni, . . .),  TODHTN 
marks  the  edge  from  ( s,tn )  to  its  first  reduced  child  (s,tni).  If  the  network  is  non¬ 
primitive,  it  marks  the  edges  to  all  the  immediate  decompositions  of  tn.  Otherwise 
the  network  is  primitive,  TODHTN  marks  all  the  possible  ending  states  (if  any). 

The  propagation  phase  itself  is  split  into  two  parts:  adding  edges,  and  propa¬ 
gating  ending  states.  When  TODHTN  adds  an  edge  from  one  problem  to  its  child, 
it  checks  to  see  if  the  child  problem  is  already  in  the  graph.  If  not,  it  adds  the 
problem  to  the  fringe.  If  the  child  is  already  in  the  current  graph,  TODHTN  marks 
all  of  the  child’s  known  endings  states  for  propagation  to  its  parent  (noting  the  edge 
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Input:  D  =  (L,  C,  O,  M)  -  an  HTN  domain 
Input:  ( s,tn )  -  an  initial  state  and  task  network 
V  <—  Fringe  <—  {(s,  tn)}; 

X(s,  tn)  0; 

while  Fringe  7^  0  &  X(s,tn)  =  0  do 
//  Pick  and  expand  a  fringe  node 
Choose  and  remove  some  (. s',tn ')  E  Fringe ; 
if  there  is  a  total  order  tri\  tnn  over  tn'  then 

J  Insert  (( s',tn ') ,  1,  ( s' ,tn\ ))  into  NewE] 
else  if  tn'  is  nonprimitive  then 

j  Insert  ((s' ,tn'),0,  (s' ,tn"))  into  NewE  for  every  decomposition  tn'  — tn"] 
else 

(s' ,tn')  is  primitive,  so  add  ((s' ,tn'),0,  se)  to  NewX  for  every  ending  state 
se  of  (s' ,  tn')] 

while  NewE  /  0  &  NewX  /  0  do 

//  Add  edges,  collect  end  states 
foreach  (v\,k,V2)  E  NewE  do 

if  V2  ^  V  then 

1  Insert  V2  into  Fringe  and  V ; 
else 

For  each  se  G  Xfo),  add  (v\,k,se)  to  NewX] 

Insert  (v\,k,V2)  into  E] 

NewE  <r-  0; 

//  Propagate  end  states 
while  NewX  /  0  do 

Choose  and  remove  some  (( sp,tnp),k,se )  from  NewX] 

Let  tn  1 , . . . ,  tnn  be  the  longest  total  order  partition  over  tnp] 

if  0  <  k  <  n  then 

|  Insert  ((sp,  tnp ),  k  +  1,  (se,  tnk+i))  into  NewE] 
else  if  se  ^  X(sp,tnp )  then 
Insert  se  into  X(sp,tnp ); 
foreach  (v,j,  (sp,tnp))  G  E  do 
[  Insert  ( v,j,se )  into  NewX] 

if  X(s,tn)  /  0  then 

return  the  preorder  traversal  of  a  subgraph  of  (V,  E)  showing  a  solution] 

else 

return  FAILURE] 

Figure  2.3:  TODHTN (D,s,tn)  A  procedure  to  explore  the  TOD  problem 
space. 
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label). 

When  processing  a  new  ending  state  se  to  propagate,  if  se  is  a  solution  to  the 
interior  part  of  a  partition  (0  <  k  <  n),  then  that  state  is  the  start  state  for  a  next 
child  in  the  partition,  (se,  trik+ 1),  and  TODHTN  marks  the  edge  to  that  problem  for 
later  addition.  Otherwise,  sp  is  an  end  state  for  ( sp,tnp ),  and  if  it  is  an  ending  state 
that  TODHTN  didn’t  already  know  about,  it  propagates  it  to  the  parents  problems 
of  (sp,tnp). 

Since  TODHTN  follows  the  definitions  of  the  TOD  problem  space  in  expanding 
nodes  from  the  fringe,  it  is  a  sound  HTN  planner.  If  the  TOD  problem  space  is  finite, 
then  TODHTN  is  complete  and  will  eventually  terminate  when  it  runs  out  of  nodes 
from  the  fringe  to  expand  and  ending  states  to  propagate.  If  the  TOD  problem 
space  is  infinite,  then  TODHTN’s  completeness  depends  upon  how  it  chooses  nodes 
out  of  the  fringe  (such  as  FIFO).  If  the  problem  is  unsolvable  and  the  problem  space 
infinite,  no  matter  how  TODHTN  chooses  it  will  never  return. 

TOPHTN  is  defined  nearly  identically  to  TODHTN,  substituting  progression 
for  decomposition. 

2.3.3  Decidability  under  problem  partitioning 

We  note  that,  since  total  order  partitions  split  task  networks  into  smaller  task 
networks  without  introducing  new  tasks,  the  TOD  and  TOP  problem  spaces  of  a 
problem  are  finite  if  the  decomposition  or  progression  problem  spaces  are  finite, 
respectively. 
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TOD  and  TOP  are  also  finite  for  a  strictly  broader  class  of  problems. 

Let  tn i , . . .  ,trik  be  the  longest  total-order  partition  of  a  task  network  tn  for  some 
number  k.  We  say  tn  is  <1  -ordered,  if  each  tn i,  for  i  —  1, . . . ,  k,  in  the  longest  total- 
order  partition  of  tn  is  either  a  singleton  or  <!-stratifiable.  An  HTN  method  ( c,tn ) 
is  <1-  ordered  if  the  task  network  tn  is  <i-ordered.  If  every  method  in  a  domain 
is  <i-ordered,  we  call  that  domain  <i-ordered,  and  if  an  HTN  planning  problem’s 
domain  and  initial  task  network  is  <i-ordered,  then  so  is  the  problem. 

Theorem  2.15  If  P  is  <\-ordered,  it  has  a  finite  TOD  problem  space. 

Proof.  Let  P  =  (D,  so,tno)  be  a  <i-ordered  HTN  planning  problem.  Since  the 
initial  task  network  tn o  is  <!-ordered  by  the  condition  of  the  theorem,  it  is  either  a 
singleton,  <i-strati£able,  or  trio  has  a  non-trivial  total  order  partition. 

The  proof  proceeds  by  showing  that  every  problem  in  TOD  problem  space  of 
P  has  a  task  network  tn,  produced  by  decomposition  over  tn0  in  s0,  that  satisfies 
at  least  one  of  the  following  conditions: 

•  tn  is  a  singleton.  Consider  a  node  with  a  singleton  task  network.  Its  task  is 
either  primitive,  i.e.,  the  node  has  no  children,  or  the  node  is  non-primitive, 
i.e.,  by  decomposition,  its  children  each  correspond  to  some  method  in  D. 

•  tn  matches  to  the  initial  network  (tn0)  or  some  HTN  method’s  task  network. 
we  have  already  shown  the  first  case.  For  the  second  case,  consider  a  non¬ 
singleton  node  (s,  tn)  with  a  task  network  tn  that  corresponds  to  some  method 
in  D ,  which  means  that  tn  can  be  produced  by  applying  an  HTN  method  to  a 


34 


nonprimitive  task  in  a  state.  Then  since  that  method  is  <i-ordered,  tn  either 
has  a  non-trivial  total  order  partition,  or  tn  is  <!-stratifiable.  In  the  first  case, 
any  children  of  ( s,tn )  have  singleton  task  networks  or  are  <i-stratifiable. 

•  tn  is  <i-stratifiable:  Note  that  in  this  case,  we  already  know  there  are  only  a 
finite  number  of  problems  reachable  from  ( s,tn )  in  the  TOD  problem  space. 

Thus,  since  the  number  of  states  in  D  is  finite,  the  TOD  problem  space  of  P 
is  finite.  □ 

We  define  <r-ordered  problems  similarly,  replacing  ^-stratification  with 
<r-stratification.  The  finiteness  of  the  TOP  problem  space  of  <r-ordered  problems 
can  be  proved  similarly. 

We  can  prove  an  infiniteness  theorem  for  TOD  and  TOP  problem  spaces  which 
is  similar  to  the  progression  space  theorem.  Here  we  pick  the  TOP  space,  since  it 
provides  a  way  to  show  the  same  for  TOD  as  well: 

Theorem  2.16  Let  P  =  ( D,s0,tn0 )  be  an  HTN  problem  where  D  has  a  method 
(c,tn)  where  c  is  a  reachable  task  name  of  tno  and  tn  is  not  <r-ordered.  If  every 
problem  in  D  is  solvable,  then  the  TOP  space  of  P  is  infinite. 

Proof.  Since  c  is  a  reachable  task  name  from  tno  and  every  problem  in  D  is  solvable, 
we  can  reach  some  problem  p  using  the  method  ( c,tn ).  tn  is  not  <r-ordered,  so  its 
longest  total  order  partition  ( tn\ , . . .  ,tnk)  has  some  non-singleton  task  network  tni 
(possibly  equal  to  tn)  which  is  not  <r-stratifiable. 

From  Theorem  2.10  we  know  that  there  is  a  chain  of  progressions  that  can 
produce  a  task  network  of  arbitrary  size  in  the  progression  space  of  any  problem 


35 


(. D,s,trii ).  Let  t\  be  the  first  progressed  task  in  this  chain.  Since  the  partition  of 
tn  was  maximal,  there  is  some  other  task  f2  that  is  not  constrained  to  come  before 
or  after  t\.  Since  the  chain  of  progressions  did  not  need  to  progress  t2  out  of  the 
task  network,  at  no  point  in  the  sequence  of  progressions  is  there  a  non-trivial  total 
order  partition  of  a  problem. 

This  means  that  the  chain  of  progressions  behaves  identically  in  the  TOP  space 
as  it  does  in  the  progression  space,  and  so  the  TOP  problem  space  of  P  is  infinite. 

□ 

The  proof  of  infiniteness  for  TOD  spaces  proceeds  similarly,  since  once  a  task 
network  has  no  non-trivial  total  order  partition,  no  sequence  of  decompositions  can 
restore  it.  As  with  progression  spaces,  <i-ordered  and  <r-ordered  are  the  broadest 
class  of  finite  problems  identifiable  without  inspecting  the  state  transition  function. 

2.3.4  Complexity  of  <1-  and  <r-ordered  problems 

Notice  that  by  the  proof  of  Theorem  2.15,  every  node  in  the  TOD  space  is 
either  a  singleton,  matches  a  task  network,  or  is  <i-stratifiable.  This  lets  us  bound 
complexity  of  planning  on  <!-stratifiable  problems: 

Theorem  2.17  plan- existence  and  k -length- plan- existence  are  in  NEXPTIME  for 
<i -ordered  problems. 

Proof.  Let  P  =  ( D,s,tn )  be  a  <i-ordered  problem.  Let  TODHTN’  be  TODHTN 
modified  so  that  in  the  main  loop,  if  it  picks  a  <i-strati£ablc  problem,  it  fully  solves 
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the  problem  for  each  possible  end  state.  An  exponential  number  of  NEXPTIME 
(Theorem  2.4)  operations  is  still  in  NEXPTIME. 

So  TODHTN’  only  adds  non-<i-stratihable  problems  to  the  fringe,  and  by  the 
proof  of  Theorem  2.15,  there  are  at  most  O  ( 2lDl)  of  them.  Given  that  a  problem 
is  only  added  to  the  fringe  once  and  we  only  propagate  end  states  from  a  given 
problem  2^  times,  there  are  at  most  an  exponential  number  of  iterations  of  the 
main  loop,  each  finishing  in  NEXPTIME.  So  <i-ordered  planning  is  in  NEXPTIME. 

□ 

By  similar  reasoning,  <r-ordered  planning  is  in  EXPSPACE. 

The  <i-ordered  and  <r-ordered  problems  both  include  what  Erol  et  al.  [1996] 
calls  totally  ordered  problems.  A  problem  is  totally  ordered  if  there  is  a  total  order 
over  the  initial  task  network  and  over  every  method’s  task  network.  Where  Erol 
et  al.  prove  that  planning  for  totally  ordered  problem  is  decidable  via  a  dynamic 
programming  argument,  we  can  repurpose  the  proof  of  Theorem  2.15  again  to  pro¬ 
vide  a  bound  on  the  size  of  TOD  and  TOP  spaces  for  totally  ordered  problems: 

Theorem  2.18  If  P,  where  D  =  ( S,C,0,M,a ),  is  totally  ordered,  then  there  are 
at  most  l  +  2lLl  •  (|M|  +  \C\  +  |0|)  vertices  in  both  the  TOD  and  TOP  problem  spaces 
ofP. 

Given  a  bound  B  on  the  number  of  vertices,  TOD-  and  TOPHTN  maintain 
O  (yB  ■  (yB  +  2lLl))  space  for  the  graph  and  map  of  vertices  to  ending  states.  Given 
that  a  vertex  is  only  added  to  the  fringe  once  and  we  only  propagate  end  states  from  a 
given  vertex  2^  times,  TOD-  and  TOPHTN  run  in  O  (H2  ■  2lLl)  time,  which  matches 
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Erol  et  al.  [1996] ’s  EXPTIME  upper  bound  for  totally-ordered  HTN  problems. 

Now  we  move  on  to  establishing  an  EXPTIME  lower  bound  on  the  complexity 
of  totally-ordered  problems.  In  games  such  as  chess,  checkers,  and  certain  versions  of 
Go,  deciding  whether  the  first  player  can  force  a  win  is  EXPTIME  complete  [Fraenkel 
and  Lichtenstein,  1981,  Robson,  1983,  1984].  In  the  remainder  of  this  section,  we 
show  how  to  encode  these  games  as  totally-ordered  HTN  planning  problems. 

We  restrict  ourselves  to  two-player,  perfect  information,  zero  sum  games  who 
have  a  polynomial  number  of  different  moves  and  whose  mechanics  are  no  harder 
than  PS  PACE.  Specifically,  checking  a  move’s  applicability,  applying  a  move,  and 
checking  win  and  lose  conditions  must  all  be  in  PS  PACE.  We  additionally  require 
that,  when  these  checks  are  translated  into  STRIPS,  which  is  PS  PACE-complete, 
every  action  has  an  inverse,  so  that  Vse2i7  (7  ( s,m )  ,  m~l)  =  s. 

With  invertible  moves,  we  can  create  HTN  tasks  which  function  as  checks 
which  ensure  some  condition  holds,  and  then  returns  to  the  state  the  task  started 
in.  We  do  this  via  palindromes.  Suppose  P  =  ( L,0,s,g )  is  a  STRIPS  problem 
with  invertible  actions.  To  represent  this  as  a  check,  we  create  an  HTN  domain 
with  the  same  operators  as  P  plus  an  operator  og  which  checks  the  goal  g.  We  add 
one  compound  task  t  to  the  domain  with  one  method  that  contains  only  og,  and  a 
method  for  each  of  the  operators  o  €  O  whose  task  networks  contain  ot  followed  by 
t  followed  by  o~l .  Thus  any  expansion  of  t  drives  down  to  a  state  that  satisfies  g, 
and  then  reverses  all  the  changes  it  made  along  the  way. 

We  will  assume  two  players,  labeled  A  and  B ,  each  with: 


38 


•  Invertible  move  operators  oq, . . . ,  dj  and  b\, . . . ,  bk  which  can  only  be  applied 
if  they  are  valid  moves. 

•  Two  checks  for  whose  turn  it  is  {turn a  and  turns)- 

•  Checks  for  winning,  losing,  and  their  negations  ( lostA ,  woua ,  not  lostA ,  etc.). 

•  A  check  for  each  of  player  BA  actions  on  whether  they  are  inapplicable 
{bi  inapplicable ,  etc). 

We  can  now  move  on  to  encoding  game  tree  search  as  a  totally-ordered  HTN  plan¬ 
ning  problem: 

Theorem  2.19  For  totally -ordered  problems,  plan- existence  is  EXPTIME -complete. 

Proof.  Let  D  =  ( L ,  C,  O,  M )  be  an  HTN  domain  with  the  tasks  described  above 
of  an  EXPTIME-complete  game,  and  let  s  be  the  game  state  to  evaluate.  We  will 
show  how  to  encode  game  tree  search  as  a  totally-ordered  HTN  problem,  so  that 
the  problem  only  has  a  solution  if  player  A  can  force  a  win. 

First,  we  add  a  new  compound  task  to  the  domain,  play,  which  will  implement 
the  game  tree  evaluation.  The  methods  for  play  will  be  recursive,  with  one  termina¬ 
tion  condition:  a  method  for  play  that  contains  the  check  ‘ won  A  ’  as  its  only  task. 
A  key  property  to  note  is  that  all  the  methods  we  introduce  for  ‘ play ’  will  return  to 
the  same  state  they  start  in,  letting  us  evaluate  multiple  potential  moves  from  the 
same  state. 

For  each  of  player  A’s  potential  move  a*,  we  add  the  method  {play,  tnai),  where 
tnai  contains  the  sequence  of  tasks  (turn a-,  not  lostA,  a?.,  piny,  If  if  is  player  A’s 
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turn  and  A  has  not  lost,  the  HTN  planner  can  pick  any  of  the  methods  corresponding 
to  one  of  H’s  moves,  apply  the  move  (if  it  is  applicable),  evaluate  lplay:  in  the  new 
state,  and  revert  the  move.  Successfully  evaluating  this  move  means  that  player  A 
can  force  a  win  in  this  state  by  picking  the  move  a*. 

We  add  one  method  (play,  ins)  for  evaluating  all  of  player  BA  moves,  where 
triB  contains  the  sequence  of  tasks  ( turns,  not  wans,  try  b\, . . .  ,try  &*,).  For  each 
compound  task  ‘ try  we  add  two  methods:  one  with  the  check  inapplicable,'1 
and  one  with  the  sequence  of  tasks  (bi,play,b~ l). 

So  if  it  is  player  B's  turn  and  B  has  not  won,  the  above  method  for  ‘ play ’  is 
the  only  method  which  can  be  applied  past  its  first  two  tasks.  The  method  then 
cycles  through  each  of  BA  valid  moves  using  Ltry  bi  to  skip  the  applicable  ones.  For 
each  applicable  move,  it  applies  the  move,  evaluates  'play'  in  the  resulting  state,  and 
reverts  the  move.  This  ensures  that  all  of  BA  potential  moves  have  been  evaluated, 
and  that  player  A  can  for  a  win  for  each  of  them. 

Let  tnpiay  be  the  task  network  containing  just  the  task  lplay\  Then  the  exis¬ 
tence  of  a  play  for  the  HTN  problem  P  =  (D,  s,  tnpiay )  implies  that  player  A  can  force 
a  win  from  state  s.  Since  the  game  was  EXPTIME-complete,  P  was  a  totally-ordered 
HTN  problem,  totally-ordered  HTN  planning  is  in  EXPTIME,  totally-ordered  HTN 
planning  is  EXPTIME-complete.  □ 

So  totally-ordered  HTN  planning  is  EXPTIME  complete,  meaning  that  TOD- 
HTN  and  TOPHTN  are  both  asymptotically  optimal  for  totally-ordered  HTN  plan¬ 
ning. 
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This  also  lends  evidence  that  TODHTN  and  TOPHTN  decide  a  larger  set  of 
problems  than  DHTN  and  PHTN,  even  though  we  have  not  shown  a  strict  separa¬ 
tion.  That  is  for  a  fixed  constant  c,  any  <]-stratifiable  or  <£-strati£able  problem 
can  be  encoded  as  a  totally-ordered  planning  problem  (a  strict  subset  of  <p  and 
<£-ordered  problems),  but,  unless  PSPACE  =  EXPTIME,  not  vice  versa. 

2.4  Practical  Considerations 

As  a  result  of  the  theoretical  analyses  presented  in  this  chapter,  a  practical 
question  arises: 

Do  existing  HTN  planning  domains  satisfy  the  finiteness  criteria  of  HTN 

problem  spaces? 

The  SHOP2  distribution  is  distributed  with  five  different  HTN  domain  models, 
namely  Logistics,  Blocks- World,  Depots,  Towers  of  Hanoi  and  Robot-Navigation.1 
All  of  these  are  both  <?.-stratifiable  and  <i-ordered,  with  Logistics  also  being  <1- 
stratifiable.  This  suggests  that  typical  HTN  domains  models  (even  complicated  ones 
such  as  Blocks- World  and  Towers  of  Hanoi,  which  encode  optimal  problem-solving 
strategies)  will  most  likely  satisfy  our  finiteness  criteria. 

Thus,  our  theoretical  and  empirical  analyses  over  HTN  problem  spaces  suggest 
the  polynomial-time  computable  conditions  for  the  finiteness  of  the  HTN  problem 
spaces,  and  the  loop-detection  tests  based  on  those  finiteness  conditions,  will  be 
practically  useful  in  at  least  two  ways: 

4  http : / / www . cs . umd . edu/ projects/ shop/ 
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•  Authors  of  HTN  domain  descriptions  will  be  able  to  use  our  theoretical  finite¬ 
ness  conditions  as  guidelines  so  as  to  obtain  guarantees  on  termination. 

•  HTN  planning  systems  incorporating  the  search  algorithms  provided  in  this 
chapter  can  determine  whether  conditions  for  finiteness  are  satisfied  during 
planning.  If  the  conditions  are  satisfied,  the  planner  can  freely  choose  any 
search  procedure  without  worrying  about  termination,  and  therefore,  com¬ 
pleteness.  Otherwise,  the  planner  can  choose  to  fall  back  onto  a  search  strat¬ 
egy  like  breadth-first  search  that  guarantees  completeness.  This  is  useful  for 
systems  such  as  SHOP2  where  depth-first  search  is  empirically  much  faster 
than  breadth-first  search. 

2.5  Discussion 

This  chapter  provides  a  classification  of  HTN  problem  spaces,  that  provides  a 
better  understanding  of  the  conditions  under  which  HTN  planning  algorithms  can 
safely  terminate  (see  Figure  2.4  for  a  summary),  as  well  as  a  number  of  complexity 
results  and  relationships  for  these  conditions  (see  Figure  2.5).  Although  this  work 
is  primarily  theoretical,  it  may  potentially  lead  to  several  practical  benefits. 

First,  there  is  reason  to  believe  that  loop-checking  tests  based  on  the  finiteness 
criteria  will  be  widely  applicable  (see  Section  2.4),  and  it  should  be  straightforward 
to  incorporate  them  into  several  existing  HTN  planners,  although  the  implementa¬ 
tion  is  left  future  work.  This  will  enable  those  planners  to  backtrack  in  cases  where 
they  otherwise  might  never  return,  thereby  enabling  the  planners  to  solve  a  larger 
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<i-ordered 


Figure  2.4:  Syntactic  containment:  Every  <i-stratifiable  problem  is  both  <r- 
stratifiable  and  <i-ordered;  every  <r-stratifiable  problem  is  <r-ordered;  and  every 
<i-ordered  problem  problem  is  <r-ordered. 


Figure  2.5:  Complexities  of  plan-existence  for  propositional  STRIPS  and  HTN  plan¬ 
ning  with  various  restrictions  on  method  structures.  Edges  represent  known  poly¬ 
nomial  encodings.  Completeness  results  for  regular  HTN  problems  and  STRIPS  are 
provide  by  Erol  et  ah  [1996]  and  Bylander  [1994] 


43 


class  of  problems.  It  might  also  make  some  planners  less  sensitive  to  the  order  in 
which  the  HTN  methods  appear  in  the  planner’s  input,  making  it  easier  to  write 
HTN  domain  descriptions. 

Second,  this  work  provides  a  useful  bound  for  the  HTN-to-PDDL  translation 
algorithm  in  Chapter  4.  That  algorithm  requires  an  upper  bound  on  the  size  of  the 
largest  task  network  in  the  HTN  progression  space,  and  if  the  user  supplies  too  low  a 
bound,  then  the  translation  algorithm  will  produce  a  classical  planning  domain  that 
is  not  a  correct  translation  of  the  original  HTN  planning  domain.  By  computing 
the  correct  bound  automatically,  it  is  easier  to  guarantee  a  correct  translation. 

Third,  the  chapter  presents  new  HTN  planning  algorithms  that  will  terminate 
in  cases  where  previous  HTN  planning  algorithms  would  not  terminate  (not  even 
with  the  incorporation  of  the  loop-checking  tests  described  above).  In  our  future 
work,  I  hope  to  implement  this  algorithm  and  test  its  performance  against  exist¬ 
ing  HTN  planners  such  as  SHOP2  and  Elkawkagy  et  al.’s  Landmark- A  ware  HTN 
planner. 

This  chapter  examines  the  algorithms  and  search  complexities  of  HTN  plan¬ 
ning.  However,  effective  search  requires  an  informed  and  efficient  heuristic.  The 
next  chapter  sets  a  hard  theoretical  boundary  on  how  informed  a  polynomial  time 
HTN  heuristic  can  be. 
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Chapter  3:  Complexities  for  Delete-Free  HTN  Planning 

Planning  has  been  shown  to  be  theoretically  intractable  in  general.  Bylan¬ 
der  [1994]  showed  that  even  the  simplest  interesting  variant  of  classical  planning  is 
PS  PACE-complete.  Hierarchical  Task  Network  (HTN)  planning  is  even  harder:  de¬ 
pending  on  the  particular  variant,  the  complexity  can  be  anywhere  from  EXPTIME 
to  undecidable  [Erol  et  al.,  1996]. 

To  combat  the  complexity  of  classical  planning,  modern  classical  planners  use 
efficiently  computable  state-based  heuristics  that  often  work  very  well  in  practice 
[Helmert,  2006,  Hoffmann  and  Nebcl,  2001,  Bonet  and  Geffner,  2001,  Nguyen  and 
Kambhampati,  2001].  The  most  influential  among  these  is  arguably  the  Relaxed 
Planning  Graph  heuristic  used  in  the  FF  planner  [Hoffmann  and  Nebel,  2001],  which 
solves  the  propositional  delete-free  version  of  the  given  problem  in  polynomial  time, 
and  computes  a  heuristic  value  based  on  that  solution.  Relaxed  planning-graph 
heuristics  have  since  been  developed  for  a  variety  of  purposes,  e.g.,  probabilistic 
planning  [Yoon  et  ah,  2007,  Teichteil-Konigsbuch  et  al.,  2010],  propositional  land¬ 
mark  generation  [Richter  and  Westphal,  2012],  metric  planning  [Hoffmann,  2003]. 

In  this  paper,  for  propositional  delete-free  HTN  planning,  we  prove  results 
about  the  complexity  of  two  well-known  decision  problems,  plan-existence  and  k- 
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Figure  3.1:  Complexity  of  plan-existence  for  propositional  delete-free  STRIPS  and 
HTN  planning  with  various  restrictions  (A:- length-plan-existence  is  NP-complete  in 
all  cases).  Arrows  represent  subclass  relationships.  The  STRIPS  results  are  from 
Bylander  [1994];  the  other  results  are  new. 

length-plan-existence,  under  various  conditions. 

Fig.  3.1  summarizes  the  results,  using  the  following  notation.  T1HTN  is  propo¬ 
sitional  HTN  planning  with  task  insertion  (see  Section  3.1  and  [Geier  and  Bercher, 
2011]);  “+pre”  (resp.  “+eff”)  means  all  preconditions  (resp.  effects)  are  positive; 
“1+pre”  (resp.  “1+eff”)  means  at  most  one  positive  and  no  negative  preconditions 
(resp.  effects).  Here  is  how  the  results  bear  on  the  feasibility  of  relaxation-based 
search  heuristics  for  HTN  planning: 

•  Even  for  very  restricted  cases,  delete-free  propositional  HTN  planning  is  NP- 
complete.  Thus  unless  P=NP,  there  is  no  direct  analogy  of  Relaxed  GraphPlan 
for  HTN  problems. 

•  If  the  HTN  planning  semantics  is  modified  to  allow  task  insertion  and  all  of 
the  preconditions  and  effects  are  positive,  then  plan-existence  is  polynomial- 
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time  computable.  Thus,  it  may  be  possible  to  use  this  or  other  relaxations  to 
develop  search  heuristics  for  HTN  planning. 

In  this  chapter,  we  consider  only  propositional  delete-free  planning  problems, 
where  operators  contain  only  positive  effects  (i.e.,  Voeode/(o)  =  0).  Deferring  to  By¬ 
lander  [1994],  we  refer  to  this  restricted  class  of  problems  as  members  of  HTN+eff. 
When  problems  are  further  restricted  to  contain  only  operators  with  positive  pre¬ 
conditions,  we  say  these  problems  belong  to  hlTN^^6.  In  the  highly  restricted  case 
where  both  the  preconditions  and  effects  of  operators  contain  at  most  a  single  pos¬ 
itive  literal,  we  say  these  problems  belong  to  HTN]^^. 

3.1  Delete-Free  Task  Insertion  HTN  Problems 

Before  we  get  to  delete-free  HTN  problems,  we  shall  first  consider  delete- 
free  versions  of  a  variant  of  HTN  planning:  HTN  Planning  with  Task  Insertion 
(TIHTN)  [Geier  and  Bercher,  2011].  In  TIHTNs,  a  problem  is  still  modeled  in  terms 
of  an  initial  state  and  a  task  network  that  needs  to  be  decomposed,  but  insertion  of 
tasks  is  now  allowed  without  requiring  them  to  be  inserted  by  the  decomposition  of 
a  compound  task  that  is  present  in  the  task  network.  As  Geier  and  Bercher  show, 
this  feature  of  TIHTNs  relaxes  HTN  planning  enough  to  regain  decidability  of  plan 
existence  even  in  cases  when  the  original  HTN  problem  remains  undecidable. 

The  following  theorem  shows  that  in  TIHTN^g6  ,  plan  existence  is  polynomial¬ 
time  computable: 

Theorem  3.1  If  P  =  ( D,s0,tn0 )  (where  D  =  (. L,C,0,M ))  is  a  Task  Insertion 
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HTN  planning  problem  with  positive  preconditions  arid  effects  (TIHTNf^),  then 
plan- existence  for  P  is  decidable  in  time  O  ( \0\2  +  |M|2) . 

Proof.  We  iteratively  apply  operators  from  O  to  So  until  we  reach  a  fixed  point  state 
s  where  no  more  operators  are  applicable,  much  like  Relaxed  GraphPlan  (taking 
O  ( | O | J )  time). 

Then  the  following  algorithm  iterates  through  the  list  of  methods  at  most  \M\ 
times  finding  a  solution  for  at  least  one  non-primitive  task  in  all  but  its  last  iteration, 
starting  from  the  non-primitive  tasks. 

1.  For  every  primitive  task  o  G  O  where  j(s,  o )  =  s,  mark  o  as  solvable. 

2.  Iterate  through  the  methods  in  M.  If  m  =  ( c,tn )  is  a  method  such  that  all 
the  tasks  names  in  tn  are  marked  as  solvable,  mark  c  as  solvable. 

3.  Repeat  line  2  if  it  marked  any  new  task  names  as  solvable. 

4.  Return  TRUE  if  all  task  names  in  trio  are  solvable,  return  FALSE  otherwise. 

Since  at  least  one  method  is  marked  in  every  pass,  this  takes  O  (|0|  +  |M|“)  time, 
resulting  in  an  overall  time  complexity  of  O  (|0|2  +  |M|2).  □ 

We  shall  now  establish  lower  bounds  on  complexities  of  both  plan-existence 
and  h- length-plan-existence  for  the  remaining  delete-free  TIHTN  planning  classes. 

Firstly,  we  note  that  (delete-free)  TIHTN  problems  can  be  encoded  as  (delete- 
free)  HTN  problems  as  follows:  given  a  TIHTN  domain  D  =  (L,C,0,M),  we 
add  for  every  t  e  C  and  o  G  O  a  method  to  M  that  decomposes  t  into  a  pair  of 
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subtasks  (o,t).  Similarly,  we  can  also  show  that  (delete- free)  STRIPS  problems  can 
be  encoded  as  (delete-free)  TIHTN  problems  by  simply  adding  a  dummy  operator  o 
with  the  goal  as  its  precondition  and  no  effects  and  letting  the  initial  task  network 
consist  of  o. 

Since  we  know  that  plan-existence  for  STRIPS+eff  and  fc-length-plan-existence 
for  both  STRIPS^0  and  STRIPS+efr  is  NP-hard  [Bylander,  1994],  it  follows  im¬ 
mediately  from  the  encoding  from  STRIPS  to  TIHTN  problems  that  plan-existence 
for  TIHTN+eff  and  fc- length-plan-existence  for  both  TIHTN^^  and  TIHTN+efj  are 
also  NP-hard. 

Now  using  the  encoding  from  TIHTN  to  HTN  problems,  we  can  similarly  lower 
bound  the  complexities  for  some  HTN  planning  problem  classes.  In  particular, 
we  can  show  that  plan-existence  for  HTN+efj  is  NP-hard  and  that  /c-length-plan- 
existence  for  both  HTN+eff  and  HTN^j6  are  NP-hard. 


Table  3.1:  Summary  of  results  from  Section  3. 


Problem 

plan-existence 

fc-length-plan-existence 

TIHTN+Pg 

P 

N  P-hard 

TIHTN+eff 

NP-hard 

N  P-hard 

htn+s? 

- 

N  P-hard 

HTN+off 

NP-hard 

N  P-hard 

Table  3.1  summarizes  the  complexity  results  from  this  section.  One  thing 
yet  to  be  done  is  to  estimate  the  complexity  of  solving  HTNt^problems.  As  we 
shall  see  in  the  following  section,  while  the  task  insertion  variant  of  this  problem 
(TIHTN^g6)  is  solvable  in  polynomial  time,  HTN+^problems  are  much  harder  to 
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solve. 


3.2  Solving  HTNj^e  Problems  is  NP-hard 

We  begin  our  analysis  on  delete-free  HTN  planning  problems  by  focusing  on 
a  restricted  case,  where 

•  Every  method  is  totally  ordered 

•  Every  method  is  regular ,  such  that  non-primitive  tasks  only  occur  as  the  last 
task  in  the  method. 

•  The  methods  are  acyclic ,  meaning  there  are  only  a  finite  number  of  solutions 
to  the  initial  problem. 

•  Every  operator  has  at  most  one  (positive)  literal  in  its  precondition  and  at 
most  one  (positive)  proposition  in  its  effect. 

We  call  the  class  of  such  HTN  planning  problems  as  HTN^lgg3  problems  for 
the  rest  of  the  paper.  In  the  following  theorem,  we  establish  the  NP-hardness  of 
plan-existence  for  HTN}  (and  thus,  for  HTN^J^6)  by  showing  a  reduction  from 
CNF-SAT: 

Theorem  3.2  Plan  existence  for  HTN\+^  planning  is  NP -hard. 

Proof.  Let  E  —  e±  A  e2  A  . . .  A  en  be  a  CNF-SAT  formula,  where  each  conjunct  is  a 
disjunction  over  a  set  of  variables  Vi, ,  vm  and  their  negations. 

To  give  an  encoding,  we  need  to  present  a  delete-free  HTN  planning  problem 
where  any  solution  implies  a  satisfying  assignment  for  E,  and  no  solution  implies 
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E  is  unsatisfiable.  The  encoding  of  E  is  the  HTN  domain  D  =  (. L ,  C,  O,  M )  and 
problem  (. D,(/},tn0 ),  all  given  below. 

Let  the  set  of  propositions  L  consist  of  two  symbols  for  each  variable,  Vi-true 
and  Vrfalse  representing  a  true  and  false  assignment  to  Vj,  respectively. 

Let  the  set  of  operators  O  consist  of  four  operators  for  each  variable  vt,  two 
for  setting  the  value  of  the  variable  and  two  for  checking  its  truth  or  negation: 

•  An  operator  set-Vi-true ,  with 

prec(set-Vi-true)  =  true, 
add(set-Vi-true)  =  {vi -true}  , 
del  [set-Vi-true)  =  0. 

•  An  operator  set-v,- false,  with 

prec(set-Vi~  false)  =  true, 
add(set-Vi- false)  =  {vr false}  , 
del  (set-Vi-  false)  =  0. 

•  An  operator  check-Vi-true,  with 

prec(check-Vi~true)  =  v^-true, 
add(check-Vi~true)  =  del  (check-Vi-true)  =  0. 

•  An  operator  check-Vc false,  with 

prec(check-Vi~  false)  =  u*-  false, 
add(check-Vi-  false)  =  del(check-Vi-  f  alse)  =  0. 
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C  and  M  consist  of  a  set  of  non-primitive  tasks  with  methods  that  first  choose 
a  variable  assignment  for  each  variable,  and  then  checks  that  each  conjunct  of  the 
expression  is  satisfied. 

For  each  variable  i C  contains  the  non-primitive  task  set-Vi.  For  %  <  m,  we 
introduce  two  methods  for  set-vf  one  which  calls  set-Vi-true  and  then  set-vi+ 1,  and 
another  which  calls  set-vr false  and  then  set-Vi+i.  For  set-vm,  we  introduce  two 
methods  as  above  but  which  call  check-e i  after  setting  the  variable  true  or  false 
(instead  of  calling  set-Vi+i). 

For  each  conjunct  e*  of  E,  C  contains  the  task  check-ei.  Since  e,:  is  a  disjunction 
of  literals,  let  h, . . . ,  4  be  the  disjuncts  of  el.  For  each  literal  lj ,  we  encode  a 
method  for  check-ei '■  if  lj  is  of  the  form  -> vi  for  some  variable  Vi,  then  the  method 
calls  check-vi- false  followed  by  check-ei+ 1.  Otherwise,  lj  is  of  the  form  vi,  and  the 
method  calls  check-vi-true  followed  by  check-ei+ The  methods  for  check-em  omit 
the  call  to  check  the  next  expression. 

The  initial  task  network  tn o  contains  a  single  task,  set-V\.  Any  primitive 
decomposition  of  the  trio  must  first  call  set-Vi-true  or  set-Vi- false  (but  not  both) 
for  each  variable,  and  then  check  that  one  literal  is  true  for  each  conjunct  in  E. 
Thus  there  exists  a  solution  to  the  HTN  problem  iff  there  is  a  satisfying  assignment 
for  the  variables  in  E. 

Since  the  encoding  is  linear  with  respect  to  the  length  of  E  and  CNF-SAT  is 
NP-hard,  delete-free  HTN  planning  is  NP-hard.  □ 
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Any  of  the  first  three  restrictions  on  method  structure  is  enough  to  place 
a  regular  HTN  planning  problem  in  a  decidable  fragment  of  the  language  [Erol 
et  ah,  1996].  This  leaves  only  two  obvious  syntactic  restrictions  that  would  make  a 
delete-free  HTN  problem  solvable  in  polynomial  time:  either  restrict  the  initial  task 
network  to  be  primitive,  or  restrict  all  operators  to  have  zero  effects. 

3.3  Showing  HTN+eff  Problems  are  in  NP 

Here  we  show  that  if  there  is  a  solution  of  length  k  to  a  delete-free  HTN  plan¬ 
ning  problem,  then  there  exists  a  polynomial  size  witness,  verifiable  in  polynomial 
time,  proving  that  there  is  exists  a  solution  of  size  k  or  smaller.  This  places  both 
plan-existence  and  k- length-plan-existence  in  NP  for  delete-free  HTN  planning. 

The  outline  of  the  proof  is  as  follows:  We  present  decomposition  trees  [Geier 
and  Bercher,  2011],  which  can  be  used  as  a  witness  that  a  task  network  is  derivable 
from  the  initial  network,  and  these  trees  can  be  verified  in  time  polynomial  in  the 
size  of  the  tree.  We  then  digress  to  show  that  deciding  whether  a  problem  has  a 
solution  when  the  primitive  tasks  that  change  the  state  are  fixed  in  advance  is  in 
NP.  Since  solutions  in  delete- free  domains  can  only  change  the  state  a  polynomial 
number  of  times,  this  lets  us  use  a  decomposition  tree  of  polynomial  width  as  part 
of  the  witness  to  the  solvability  of  HTN+eff  problems.  Finally,  we  also  provide  a 
polynomial  bound  on  the  height  of  a  decomposition  tree  necessary  to  show  that 
problem  is  solvable. 
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3.3.1  Decomposition  Trees 


Geier  and  Bercher  [2011]  introduced  the  idea  of  decomposition  trees ,  which  is 
a  representation  of  how  the  initial  compound  task  cj  can  be  transformed  to  a  task 
network  tn  via  a  sequence  of  decompositions.1  This  section  presents  their  definitions 
below,  modified  slightly  to  suit  our  purposes. 

Given  a  planning  problem  P,  a  decomposition  tree  g  =  (T,  E,  -<,a,  (3)  is  a 
five-tuple  satisfying  the  following  properties: 

•  (T,  E )  is  a  tree  with  nodes  T  and  directed  edges  E  pointing  towards  the  leaves; 

•  -<  is  a  partial  order  defined  over  T ; 

•  a:T->CUOisa  labeling  function  that  labels  the  nodes  in  T  with  task 
names; 

•  (3  is  a  labeling  function  that  labels  each  inner  node  with  a  method  m  =  (c,  tnm) 
and  an  isomorphism  from  tnm  to  the  children  of  that  node. 

Moreover,  T(g)  is  defined  to  refer  to  the  tasks  of  g  and  ch(g,t)  to  refer  to  the 
direct  children  of  t  G  T(g)  in  g. 

The  following  definition  states  the  conditions  under  which  a  decomposition 

tree  encodes  a  decomposition  of  the  initial  task  network.  A  decomposition  tree 
1Note  that  the  restriction  for  having  a  single  task  for  the  initial  task  network  of  an  HTN  planning 
problem  is  only  for  the  sake  of  simplifying  the  exposure  of  our  theoretical  results;  the  definitions 
and  theorems  in  this  section  can  be  adapted  to  work  without  this  restriction  by  generalizing  the 
notion  of  decomposition  trees,  described  below,  to  decomposition  forests. 
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g  =  (T,  E,  -<,  a,  f3)  is  valid  with  respect  to  a  planning  problem  P  =  (. D ,  s0,  cj)  if  and 
only  if  the  root  node  of  g  is  labeled  with  the  initial  task  name  Cj  and  for  any  inner 
node  t,  where  (3(t)  =  ((c,tnm),  /),  the  following  conditions  hold: 

1.  a(t)  =  c, 

2.  /  is  a  valid  isomorphism  of  the  task  network  induced  in  g  by  ch (g,  t )  and  tnm ; 
i.e. 

^  |ch(g,t)  j  &  |ch(g,t) )  — 

3.  Vt'  G  T,  d  G  ch (^7,  t) ,  it  holds  that 

(a)  if  t  -<  t'  then  d  -<  t'\ 

(b)  if  t'  <t  then  t'  -<  d . 

4.  there  are  no  other  ordering  constraints  in  -<  other  than  those  demanded  by 
conditions  2  and  3. 

Informally,  the  above  conditions  capture  the  following  checks  for  each  inner 
node  t:  condition  1  verifies  the  applicability  of  the  method  m  =  f3(t)  that  t  is  labeled 
with;  condition  2  verifies  that  m’s  task  network  is  correctly  represented  in  the  tree; 
condition  3  ensures  that  the  ordering  constraints  are  inherited  correctly  after  the 
application  of  m;  and  condition  4  ensures  the  minimality  of  -<. 

The  definition  of  a  decomposition  tree  and  its  validity  to  an  HTN  planning 
problem  is  identical  to  Geier  and  Bercher’s  definition,  save  for  the  addition  of  the 
explicit  isomorphism  at  each  inner  node  t,  mapping  ch  (g,t)  to  the  subtask  network 
of  the  method  applied  at  t.  This  modification  is  made  so  that  the  validity  of  a 
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decomposition  tree  can  be  checked  in  time  polynomial  in  the  size  of  the  tree2.  Note 
that  the  theoretical  results  in  Geier  and  Bercher  [2011]  still  hold  unchanged  even 
with  these  modifications.  This  is  an  important  point  as  we  shall  be  using  their 
theorems  (which  they  proved  under  their  definition)  in  our  proofs. 

Note  that  the  leaves  of  a  decomposition  tree  g  form  a  task  network,  which  is 
called  the  yield  of  g.  Formally,  the  yield  of  a  decomposition  tree  g  =  (T,  E,  -<,  a,  f3) 
is  a  task  network  defined  as  follows.  Let  T'  C  T  be  the  set  of  all  leaf  nodes  in  g. 
Then,  yield(^)  =  (T',  -<  |T/,a|T/). 

Geier  and  Bercher  [2011]  use  the  above  definitions  to  prove  the  following  useful 
property  of  valid  decomposition  trees: 

Theorem  3.3  Given  a  planning  problem  P  =  (. D,s0,ci ),  the  following  holds  for 
any  task  network  tn  e  TNcuo-  There  exists  a  valid  decomposition  tree  g  with 
yield (p)  =  tn  if  and  only  if  cj  —t*D  tn. 

In  other  words,  the  reachability  of  tn  from  c.j  via  a  sequence  of  method  de¬ 
compositions  can  be  proved  by  providing  a  valid  decomposition  tree  for  the  problem 
P  whose  yield  is  tn.  This  property,  as  we  shall  see  later,  will  be  instrumental  in 
proving  that  delete- free  HTN  planning  is  in  NP. 

Given  a  decomposition  tree  g  —  (T,  E,  a ,  (3)  and  a  node  t  E  T,  the  subtree 
of  g  induced  by  t,  written  as  g[t],  is 

g[t]  =  (■ T',E',-<  |t',  P\t>), 

2  Since  graph  isomorphism  is  not  known  to  be  in  P,  this  would  not  be  possible  without  our 
modification. 
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where  (T',  E ')  is  the  subtree  of  (T,  E)  rooted  at  t. 


Definition  3.4  Let  g  =  (T,  E,  -<,a,  (3)  be  a  decomposition  tree  and  ti,tj  G  T  be 
two  nodes  of  g.  The  result  of  the  subtree  substitution  of  ti  with  tj  on  g,  written  as 
g[tj  <—  tf],  is  given  as  follows: 

•  If  L  is  the  root  node  of  g,  then  g[ti  <—  tj]  =  g[tf . 

•  Otherwise,  g[L  G-  tj]  =  (T',E',-<  \t>,  cx\t>,  P\t'),  with 

-  T'  =  (T\T(g[U]))  UTtsM), 

-  E'  —  E\t>  U  {(p,  tj)},  where  p  is  the  parent  node  of  ti  in  g. 

Note  that  this  operation  in  general  will  not  lead  to  valid  decomposition  trees. 
However,  if  applied  under  the  right  conditions,  the  result  of  the  subtree  substitution 
can  still  describe  valid  decompositions  as  described  by  the  following  result  [Geier 
and  Bercher,  2011]: 

Theorem  3.5  Let  g  =  (T,  E,  -<,  a,  /3)  be  a  valid  decomposition  tree  for  an  HTN 
planning  problem  P.  If  we  are  given  two  nodes  ti  G  T,tj  G  T(g[ti])  such  that 
a(ti )  =  a(tj),  then  g[L  <—  tj]  is  also  a  valid  decomposition  tree  for  P. 

In  other  words,  if  ti  and  tj  map  to  the  same  task  names  and  tj  is  a  descendant 
of  ti  in  g ,  then  replacing  ti  (and  its  subtree)  with  tj  (and  its  subtree)  still  results  in 
a  valid  decomposition  tree.  This  technique  can  therefore  be  used  to  eliminate  cyclic 
decompositions  from  a  tree  while  still  retaining  validity. 
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3.3.2  Forming  a  witness  to  the  solvability  of  an  HTN  problem 

We  are  going  to  use  decomposition  trees  to  show  that  delete- free  HTN  planning 
is  in  NP.  Note  that  if  a  valid  decomposition  tree’s  yield  is  primitive  and  executable, 
then  we  can  use  the  tree  as  a  checkable  proof  that  its  problem  is  solvable.  However, 
even  in  the  restricted  case  where  none  of  the  operators  have  an  effect,  the  minimal 
solution  size  (measured  in  the  number  of  tasks)  may  still  be  exponential.  So  we 
need  to  be  able  to  present  a  witness  that  includes  both  a  tree  with  a  non-primitive 
yield,  and  a  polynomial  size  proof  that  some  expansion  of  that  yield  is  executable. 

Definition  3.6  Let  P  =  ( D,s0,tn0 )  be  an  HTN  planning  problem,  where  D  = 
(. L ,  C,  O,  M )  and  tn0  =  (T0,  -<0,  cto)-  3  state-transition  preserving  solution  for  P  is 
one  in  which  the  only  state- changing  actions  are  the  ones  that  were  already  in  trio, 
i.e.,  it  is  a  primitive  task  network  tn  such  that: 

•  trio  ~^*d  t'n,  where  tn  =  (T,  a) 

•  tn  has  an  executable  ordering  over  its  tasks  (t\, . . .  ,tn,  executing  over  the  states 
So  to  snJ 

•  Ifti  £  To  then  Sj_ i  =  7  (s*_i,  a(L))  = 

Given  a  sequence  of  states,  a  solution  table  for  finding  a  state-transition  pre¬ 
serving  solution  consists  of  a  row  for  each  combination  of  start  state,  end  state,  and 
task  name.  Each  row  in  a  solution  table  has  a  value,  defined  as  follows: 

•  For  each  row  with  a  primitive  task  name,  the  value  of  that  row  is  1  if  the  ground 
instance  of  the  operator  for  that  primitive  task  name  is  both  applicable  in  any 


state  between  the  start  state  and  end  state,  inclusively,  and  the  operator  does 
not  change  said  state.  Otherwise,  the  value  of  the  row  is  oo. 

•  For  each  row  with  a  non-primitive  task  name,  we  associate  a  method  used  to 
decompose  the  task,  and  a  set  of  pointers  back  into  the  table  supporting  that 
the  method  is  executable  (without  changing  the  state)  between  the  start  and 
end  state  for  the  row.  The  value  for  the  row  is  then  the  sum  of  the  values  of 
its  supporting  rows. 

We  can  check  the  table  by  first  checking  the  primitive  entries  of  the  table,  and 
then  repeatedly  scanning  the  table  to  find  rows  whose  supports  have  already  been 
checked.  This  leads  into  the  following  lemma: 

Lemma  3.7  Both  the  plan-existence  and  the  k-length-plan-existence  problems  for 
finding  a  state-transition  preserving  solution  are  in  NP. 

Proof.  Let  P  =  (. D ,  so,  tn o)  be  an  HTN  planning  problem  where  D  =  (L,  C,  O,  M), 
tn0  =  (T0,  -<o,  «o)  such  that  P  has  a  state-transition  preserving  solution. 

By  definition,  in  any  state-transition  preserving  solution,  only  the  primitive 
tasks  already  in  tno  may  change  the  state.  So  given  a  fixed,  executable  ordering  over 
the  primitive  tasks  of  tn0  and  the  states  associated  with  that  ordering  (s0, . . . ,  sn), 
the  decompositions  of  non-primitive  tasks  in  tn o  interact  with  each  other  only  in 
what  states  they  start  and  end  on  (constraining  the  end  and  start  states,  respectively, 
of  tasks  required  to  come  before  or  after).  Start  and  end  states  for  a  task  determine 
what  decompositions  (if  any)  are  executable  over  that  sequence  of  states.  This  lets 
us  construct  a  solution  table  as  described  above. 
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Once  the  solution  table  is  constructed,  a  witness  to  the  solvability  of  P  (i.e.,  a 
witness  that  there  exists  a  state-preserving  solution  for  P)  consists  of  a  total  order 
over  the  primitive  tasks  of  trio,  a  solution  table  described  above  for  the  sequence 
of  states  traversed  by  those  primitive  tasks,  and  a  set  of  pointers  into  the  table  for 
each  non-primitive  task  in  trio ■  The  value  of  the  solution  is  the  sum  of  the  primitive 
tasks  in  the  row  of  the  solution  table  that  holds  trio,  plus  the  sum  of  the  values 
sizes  of  the  supporting  table  entries.  Since  the  validity  of  the  ordering  and  table 
are  verifiable  in  polynomial  time,  both  plan-existence  and  h-length-plan-existence 
for  finding  a  state-transition  preserving  solution  are  both  in  NP.  □ 

We  can  now  use  a  decomposition  tree  as  a  proof  that  an  HTN  problem  is 
solvable,  even  if  the  yield  of  that  tree  is  non-primitive: 

Definition  3.8  Let  tn  be  a  task  network,  g  be  valid  decomposition  tree  of  tn,  and 
stp  be  a  witness  that  the  yield  of  g  has  a  state-transition  preserving  solution.  Then, 
a  witness  to  the  solvability  of  an  HTN  problem  P  =  ( D ,  s,  tn)  is  the  pair  ( g ,  stp)  of 
a  valid  decomposition  tree  g  oftn  with  stp. 

Since  checking  the  validity  of  a  tree  is  polynomial  in  the  size  of  the  tree,  and 
checking  the  witness  that  the  yield  of  the  tree  has  a  state-transition  preserving 
solution  is  polynomial  in  the  size  of  the  yield  and  the  number  of  task  names,  it 
follows  that  the  combined  witness  is  also  in  P.  Furthermore,  note  that  every  solvable 
HTN  planning  problem  has  a  witness,  even  non-delete-free  problems.  However,  the 
existence  of  a  polynomial-sized  witness  is  only  likely  in  delete-free  planning,  where 
a  fix-point  state  is  reachable  in  a  polynomial  number  of  actions.  In  the  remaining 
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sections,  we  show  that  delete-free  HTN  planning  problems  always  have  a  witness  of 
polynomial  size. 

3.3.3  Bounding  the  breadth  of  the  witness  tree 

Given  a  delete-free  HTN  problem  and  its  witness,  ( g,stp ),  we  know  there  are 
at  most  \0\  primitive  tasks  which  change  the  state  in  any  execution  of  the  yield  of 
g,  where  O  is  the  set  of  operators.  We  now  show  how  to  restrict  a  decomposition 
tree  to  its  minimal  valid  subtree  that  contains  those  operators. 

Definition  3.9  (Saplings)  Given  a  tree  g  =  (T,E,  -<,a,/3)  and  a  set  of  tree  nodes 
S  C  T ,  let  T'  be  the  set  of  nodes  along  any  path  from  a  node  in  S  to  the  root  of  g 
(inclusively)  and  the  siblings  of  each  and  every  node  along  the  path.  Formally,  T'  is 
the  smallest  subset  of  T  such  that: 

•  S  CT' 

•  VeTt'  G  V  A  (t,  t!)  G  E  ==►  t  G  V 

•  £  T')  A  G  E  =>•  t2  G  T' 

Let  T"  contain  the  inner  nodes  ofT'.  The  the  sibling-augmented  path  tree  or 
S-sapling  of  T  is  the  decomposition  tree  given  by  the  tuple: 

(' T E\t>,  ol\T’,  P\t") 

Proposition  3.10  Given  a  tree  g  =  (T,  -<,  a ,  (3)  and  a  set  S  C  T,  then  the  S-sapling 
of  g  is  a  valid  decomposition  tree. 
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Proof.  Any  subtree  of  g  containing  the  root  satisfies  all  but  condition  3  of  definition 
a  valid  decomposition  tree.  Since  the  construction  of  a  sapling  either  preserves  all 
children  of  node  or  none  of  them,  condition  3  also  holds.  □ 

Given  a  witness  (g,  stp )  for  a  delete-free  problem,  we  can  create  a  sapling  using 
just  the  primitive  tasks  that  change  the  state. 

Lemma  3.11  Let  ( g ,  stp)  be  a  witness  that  a  delete-free  HTN problem  P  =  ( D ,  s0,  tn0 ) 
with  domain  D  =  (L,C,0,  M)  is  solvable.  Let  (T,  -<,a)  =  yield(g),  and  let  S  C  T 
be  the  set  of  tasks  that  change  the  state  in  the  order  specified  by  stp.  Then  if  g'  is  the 
S -sapling  of  g,  there  exists  a  witness  stp'  such  that  the  yield  of  g'  has  state-transition 
preserving  solution  of  the  same  size  or  smaller  than  the  yield  of  g. 

Proof.  Given  that  stp  is  the  witness  that  g  has  a  state-transition  preserving  solu¬ 
tion,  let  (<,B,R)  =  stp,  where: 

•  <  is  (ti, . . . ,  tn)  which  is  the  total  ordering  over  the  primitive  tasks  in  the  yield 
of  g.  Let  si, . . .  ,sm  be  the  distinct  states  that  sequence  produces  (omitting 
repeated  states). 

•  B  is  the  solution  table  for  the  sequence  si, . . . ,  sm.  Assume  WLOG  that  B  is 
optimal,  giving  the  best  possible  cost  for  each  entry  in  B. 

•  R  is  the  set  of  pointers  into  B  for  non-primitive  tasks  in  the  yield  of  g. 

Let  S  be  the  set  of  primitive  tasks  that  change  the  state,  or 
S  =  {ti  G  yielding)  |  i  ^  st } ,  and  let  g'  be  the  S-sapling  of  g. 
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Now  we  provide  a  witness  that  g'  has  a  state-transition  preserving  solution. 
Let  <’  be  the  same  ordering  as  <  restricted  to  tasks  in  S.  Since  tasks  in  <  but 
not  in  <'  did  not  change  the  state,  an  execution  of  <'  produces  the  same  sequence 
Si, . . . ,  sm  of  distinct  states  that  <  did,  and  so  we  can  reuse  the  same  solution  table 
B. 

For  the  set  of  supports,  any  task  t  in  the  yield  of  g'  which  was  not  in  the  yield 
of  g  must  have  children  in  the  yield  of  g  which,  under  the  given  ordering  <,  were 
all  either  primitive  tasks  which  did  not  change  the  state  or  were  non-primitive  with 
state-transition  preserving  expansions  with  entries  in  R.  So  B,  the  solution  table, 
must  have  an  entry  for  Si,Sj,t  with  finite  value,  where  st  and  Sj  are  the  first  state 
and  last  in  the  sequence  So, ...  ,sm  where  either  primitive  descendant  was  executed 
or  the  first  state  used  in  R  for  a  non-primitive  descendant.  So  we  can  construct  a 
new  set  of  supports  R'  using  the  above  method  for  any  task  in  the  yield  of  g'  but 
not  in  R,  and  directly  using  the  entry  from  R  otherwise. 

So  stp'  =  (<',  B ,  R')  is  a  witness  that  g'  has  a  state-transition  preserving 
solution.  Moreover,  since  B  remains  the  same  and  R'  was  calculated  from  B  and  R , 
stp'  must  indicate  that  g'  has  a  solution  in  B  with  the  same  or  lower  value  as  g.  □ 

3.3.4  Bounding  the  height  of  the  witness  tree 

The  above  lemma  lets  us  take  any  witness  ( g ,  stp)  to  a  problem’s  solvability 
and  construct  a  new  witness  which  is  composed  of  a  polynomial  number  of  paths 
to  the  root  g  (plus  siblings).  This  is  not  quite  enough  to  show  that  delete-free 
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planning  is  in  NP,  since  those  paths  may  not  be  polynomial  in  length.  However, 
in  those  cases,  we  can  use  a  variant  of  the  pumping  lemma  [Comon  et  ah,  2007]  to 
produce  a  new  witness  with  polynomially-bounded  length  paths: 

Theorem  3.12  Let  P  =  (D,  so ,  trio)  (where  D  =  (L,  C,  O,  M ) )  be  a  solvable  delete- 
free  HTN  ■planning  problem,  with  P  having  a  minimal  solution  size  of  k  .  Then  there 
exists  a  witness  (g,  stp )  that  P  has  a  solution  size  of  <  k,  with  \T(g)\  <  m ■  \C\  ■  \0\~ , 
where  m  is  the  size  of  the  largest  task  network  in  M . 

Proof.  If  P  is  solvable,  there  exists  a  tree  gp  with  an  executable,  primitive  yield 
of  optimal  cost  k.  Let  (g,  stp )  be  the  Stapling  witness  as  constructed  above  in 
lemma  3.11,  where  S  is  the  set  of  tasks  in  the  witness  that  change  the  state.  Then 
( g,stp )  is  a  witness  that  P  has  a  solution  of  size  <  k. 

Suppose  g  has  a  height  that  is  greater  than  \C\  ■  \()\.  Since  g  is  constructed 
from  a  series  of  paths  from  nodes  to  the  root,  this  means  that  there  is  some  path 
from  a  node  in  S  to  the  root  of  that  length. 

Let  ti, ...  ,tn  be  the  tasks  along  that  path.  Since  that  path  is  joined  at  most 
| S\  —  1  times  by  other  paths  from  S  to  the  root  (|Sj  <  0|)  and  since  there  are  only 
\C\  task  names  to  assign,  there  must  be  some  segment  L, . . .  ,tj  between  joins  such 
that  a(ti)  =  a(tj),  and  no  descendants  of  t,  not  on  the  path  to  tj  has  a  descendants 
that  is  in  S. 

Since  no  descendants  of  t,t  that  are  not  also  a  descendants  of  tj  are  in  S,  then 
all  of  those  descendants  must  have  a  state-transition  preserving  solution  under  stp' . 
Let  g'  —  g  [ti  <—  tj]  be  the  tree  obtained  by  substituting  tj  for  tt.  Since  we  only 
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removed  tasks  which  did  not  change  the  state,  the  yield  of  g'  is  a  strict  subset  of  the 
yield  of  g.  So  we  can  create  a  witness  stp'  that  g'  has  a  state-transition  preserving 
solution  by  restricting  the  set  of  supports  in  stp  to  the  tasks  remaining  in  the  yield 
of  g' .  That  solution  must  have  a  cost  strictly  less  than  k.  This  would  violate  our 
assumption  k  was  the  minimal  solution  size. 

So  g  must  have  a  height  that  is  less  than  or  equal  to  \C\  •  |0|.  □ 

Since  we  can  always  find  a  polynomial  sized  witness  to  the  minimal-sized 
solution,  this  means  that  finding  k- size  solution  (or  any  solution)  to  a  delete-free 
HTN  problem  (HTN+efj)  is  in  NP.  Given  that  both  plan-existence  and  /e-length- 
plan-existence  are  NP-hard  for  HTN+efj,  the  last  of  our  results  is  trivial: 

Theorem  3.13  For  HTN+eff,  both  plan- existence  and  k -length-plan- existence  are 
HP -complete. 

From  this  theorem  and  the  subclass  relationships  shown  in  Figure  3.1,  the  other 
classes  considered  in  this  paper  fall  in  NP  as  well  and  are  thus  NP-complete  (with 
the  exception  of  plan-existence  for  TIHTN^g^0,  of  course).  Table  3.2  summarizes 
our  final  set  of  results  for  delete-free  HTN  planning. 

3.4  Discussion 

In  classical  planning,  relaxing  the  planning  problem  by  removing  negative 
preconditions  and  effects  has  been  quite  useful  in  the  development  of  efficiently 
computable  search  heuristics.  This  chapter  shows  that  this  relaxation  will  not — by 
itself — produce  efficiently  computable  HTN  planning  heuristics,  because  the  relaxed 
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Tabic  3.2:  Summary  of  results  after  Section  3.3. 


Problem 

plan-existence 

h-length-plan-existence 

TIHTN+Pg 

P 

N  P-complete 

TIHTN+eff 

NP-complete 

N  P-complete 

HTN«;= 

NP-complete 

N  P-complete 

htn+.„ 

NP-complete 

N  P-complete 

problems  are  NP-hard.  Thus  the  development  of  search  heuristics  for  HTN  planning 
will  require  a  new  kind  of  problem  relaxation. 

The  solution  tables  that  we  used  in  the  proof  of  Lemma  3.7  are  a  data  structure 
similar  to  planning  graphs,  and  it  might  be  possible  to  use  them  as  a  foundation 
to  develop  new  heuristics  and  search  techniques  for  generating  compact  witnesses. 
Such  witnesses  could  be  used  to  provide  heuristic  estimates  of  relaxed  plan  length. 
Furthermore,  a  solution  table  also  exhibit  similarities  to  that  of  a  chart  in  chart 
parsing  [Kay,  1986,  Earley,  1986,  Charniak  et  al.,  1998,  Ji,  1993],  in  both  the  way 
the  solution  table  is  generated  and  structured.  This  suggests  that  it  might  be 
possible  to  use  the  techniques  to  generate  efficient  parse-trees  in  chart  parsing  for 
witness  inference. 

Another  approach  may  be  to  combine  relaxed  planning  graphs  with  a  relax¬ 
ation  of  the  constraints  that  HTN  planning  formalisms  impose  on  the  search  process. 
For  example,  our  results  show  that  efficiently  computable  HTN  planning  problems 
can  be  produced  by  removing  negative  preconditions  and  effects,  and  also  allowing 
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task  insertion  (i.e. ,  allowing  the  application  of  any  executable  operator,  regardless  of 
whether  or  not  it  is  reachable  by  some  decomposition).  We  suspect  that  this  might 
relax  the  problem  too  much  for  the  heuristic  values  to  be  useful.  But  we  think  it 
may  be  possible  to  develop  more  accurate  yet  efficiently  computable  heuristics  by 
developing  a  principled  compromise,  e.g.,  by  restricting  the  inserted  tasks  to  those 
available  in  some  decomposition  of  the  current  task.  This  would  be  an  interesting 
topic  for  future  research. 

Given  that  full-semantics  delete-free  HTN  planning  is  out  of  reach  for  an  HTN 
heuristic,  one  might  question  whether  less  informed  heuristics  will  provide  effective 
search  guidance.  The  next  chapter  shows  totally-ordered  <r-stratihable  problems 
can  be  translated  into  classical  planning  problems.  This  opens  up  the  use  of  classical 
planning  heuristics  for  HTN  planning. 
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Chapter  4:  HTN  Planning  via  translation  to  PDDL 


I  show  that  HTN  planning  knowledge,  if  it  satisfies  some  restrictions,  can 
automatically  be  translated  into  PDDL,  and  that  even  small  amounts  of  such 
knowledge  can  greatly  improve  a  classical  planner’s  performance.  In 
particular: 

•  Section  4.2  describes  how  to  translate  a  restricted  class  of  HTN  methods  and 
operators  into  PDDL.  We  provide  theorems  showing  that  our  translation  is 
correct,  that  its  time  and  space  complexity  are  both  linear,  and  that  it  can  be 
used  even  on  partial  HTN  models  of  a  domain  (which  can  be  much  easier  to 
write  than  full  HTN  models). 

•  Experiments  in  Section  4.4  show  that  by  translating  partial  HTN  models  into 
PDDL,  we  can  substantially  improve  a  classical  planner’s  performance.  In 
experiments  with  the  well-known  Fast-Forward  (FF)  planner  [Hoffmann  and 
Nebel,  2001]  on  more  than  3500  planning  problems,  the  translated  knowledge 
improved  FF’s  running  time  by  several  orders  of  magnitude,  and  enabled  it  to 
solve  much  larger  planning  problems  than  it  could  otherwise  solve. 


4.1  Basic  Definitions  and  Notation 


Chapter  1  introduced  propositional  HTN  and  classical  planning  formalisms 
for  ease  of  analysis.  Most  planning  implementations,  however,  are  set  to  work  over 
fragments  of  first-order  logic.  This  section  presents  formalisms  defined  over  function- 
free  first-order  logic  for  classical  planning  and  totally-ordered  HTN  planning  that 
closely  match  the  PDDL  planning  language  and  SHOP  HTN  language,  respectively 
[Fox  and  Long,  2003,  Nau  et  al.,  2001]. 

4.1.1  Classical  Planning 

Our  definitions  for  classical  planning  are  based  on  the  ones  in  Ghallab  et  al. 
[2004], 

Let  L  be  the  set  of  all  literals  in  a  function-free  first-order  language.  A  state  is 
any  set  of  ground  atoms  of  L.  A  classical  planning  problem  is  a  triple  P  =  (s0,  g ,  O), 
where  So  is  the  initial  state,  g  is  the  goal  (a  set  of  ground  literals  of  L),  and  O  is  a 
set  of  operators.  Each  operator  o  G  O  is  a  triple 

o  =  (name(o),  precond(o),  effects  (o)), 

where  narne(o)  is  o’s  name  and  argument  list,  and  precond(o)  and  effects(o)  are  sets 
of  literals  called  o’s  preconditions  and  effects.  An  action  a  is  a  ground  instance  of  an 
operator.  If  a  state  s  satisfies  precond(o;),  then  a  is  executable  in  s,  producing  the 
state  j(s,a)  —  (s  —  (all  negated  atoms  in  effects(a)})  U  (all  non-negated  atoms  in 
effects(a)}.  A  plan  is  a  sequence  tt  =  (oq, . . . ,  an)  of  actions,  n  is  a  solution  for  P 
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if,  starting  in  s0,  the  actions  are  executable  in  the  order  given  and  the  final  outcome 
is  a  state  sn  that  satisfies  g. 

4.1.2  TSTN  Planning 

Ghallab  et  al.  [2004]  describes  a  restricted  case  of  HTN  planning  called  Total- 
order  Simple  Task  Network  planning,  which  we’ll  abbreviate  as  TSTN  Planning. 
The  definitions  are  as  follows. 

A  task  is  a  symbolic  representation  of  an  activity.  Syntactically,  it  is  an  ex¬ 
pression  t  =  t(x i, . . . ,  xq)  where  t  is  a  symbol  called  r’s  name,  and  each  ay  is  either 
a  variable  or  a  constant  symbol.  If  t  is  also  the  name  of  an  operator,  then  r  is 
primitive;  otherwise  r  is  nonprimitive.  Intuitively,  primitive  tasks  can  be  instanti¬ 
ated  into  actions,  and  nonprimitive  tasks  need  to  be  decomposed  (see  below)  into 
subtasks. 

A  method  is  a  prescription  for  how  to  decompose  a  task  into  subtasks.  Syn¬ 
tactically,  it  is  a  four-tuple 

m  =  (name(ro),  task(m),  precond(m),  subtasks(m)), 

where  name(m)  is  m’s  name  and  argument  list,  task(m)  is  the  task  m  can  decompose, 
precond(m)  is  a  set  of  preconditions,  and  subtasks(m)  =  (t\, . ,tj)  is  the  sequence 
of  subtasks. 

A  TSTN  planning  problem  is  a  four-tuple  P  =  (s0,T0,O,  M),  where  s0  is  an 
initial  state,  O  is  a  set  of  operators,  T0  is  a  sequence  of  ground  tasks  called  the 
initial  task  list,  and  M  is  a  set  of  methods. 
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If  T0  is  empty,  then  P’s  only  solution  is  the  empty  plan  7 r  =  (),  and  7r’s 
derivation  (the  sequence  of  actions  and  method  instances  used  to  produce  n)  is 
5  =  ().  If  To  is  nonempty  (i.e.,  T0  =  {ti, . . .  ,tk)  for  some  k  >  0),  then  let  T'  = 
(f2, . . .  ,tk).  If  t\  is  primitive  and  there  is  an  executable  action  a  with  name(a)  =  t,\ , 
then  let  Si  =  7(s0,  a).  If  P'  =  (si,  T',  O,  M)  has  a  solution  n  with  derivation  5,  then 
the  plan  a  •it  is  a  solution  to  P  (where  •  is  concatenation)  whose  derivation  is  a* 5. 
If  t\  is  nonprimitive  and  there  is  a  method  instance  m  such  that  task(m)  =  t\,  and 
if  s0  satisfies  precond(m),  and  if  P'  =  (si,  subtasks(m)  •  T',  O,  Ad )  has  a  solution  7r 
with  derivation  6,  then  n  is  a  solution  to  P  and  its  derivation  is  m  •  6. 

4.1.3  Using  TSTN  Planners  for  Classical  Planning 

To  use  a  TSTN  planner  in  a  classical  planning  domain  D  (i.e.,  a  set  of  classical 
planning  problems  that  all  have  the  same  operator  set  O ),  the  usual  approach  is 
augment  D  with  a  set  M  of  methods  and  a  way  to  translate  each  classical  goal  g 
into  a  task  list  T59.  This  maps  each  classical  planning  problem  P  =  (so,g,0)  in  D 
into  a  TSTN  planning  problem  P'  =  (s0,  T$,  O,  Ad).  The  mapping  is  correct  if  P'  is 
solvable  whenever  P  is,  and  if  the  solutions  for  P'  are  also  solutions  for  P.  Since 
the  objective  is  for  P'  to  have  a  small  search  space,  the  set  of  solutions  for  P'  may 
be  much  smaller  than  the  set  of  solutions  for  P. 

In  the  above  mapping,  we  will  say  that  M  is  O- complete  if  every  operator  in 
o  G  O  is  mentioned  in  Ad,  i.e.,  at  least  one  method  in  Ad  has  a  subtask  that  is  an 
instance  of  narne(o). 
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4.2  Translating  TSTN  to  Classical 


Let  P  =  (s0,  T,  O,  M )  be  a  TSTN  planning  problem,  and  suppose  T0  is  a 
correct  translation  (as  defined  above)  of  a  classical  goal  g.  We  now  describe  how 
to  translate  P  (if  a  restriction  holds)  into  a  classical  planning  problem  trans(P)  = 
(sq,  g,  O')  that  is  equivalent  to  P  in  the  following  sense:  as  we’ll  show  in  Section  4.3, 
there  is  a  one-to-one  mapping  from  P’s  solution  derivations  to  trans{P)' s  solutions. 

4.2.1  Preliminaries 

We  begin  by  introducing  a  restriction.  For  every  solution  n  of  P,  let  the  non¬ 
tail  height  of  tt  be  the  number  of  levels  of  method  decomposition  used  to  produce 
7r,  ignoring  tail  decomposition  (i.e. ,  decomposition  of  the  last  task  in  a  task  list). 
Then  either  we  need  to  extend  the  planning  language  to  include  function  symbols,1 
or  else  we  must  be  given  an  upper  bound  H  on  the  non-tail  height  of  all  solutions 
of  P.  The  non-tail  height  is  equivalent  to  the  height  of  the  minimum  stratification 
in  totally-ordered  <r-stratifiablc  problems. 

We  need  the  above  restriction  in  order  to  implement  a  symbolic  representa¬ 
tion  of  a  numeric  counter,  to  keep  track  of  the  current  number  of  levels  of  task 

decomposition.  Here  is  how  to  implement  the  counter  when  H  is  given:2 

1PDDL  includes  this  extension,  but  traditional  formulations  of  classical  planning  (e.g.,  [Ghallab 
et  al.,  2004])  do  not. 

2If  H  is  not  given  but  the  planning  language  contains  function  symbols,  we  can  instead  use  an 
unbounded  number  of  ground  terms  1,  next(l),  next(next(l)), .... 
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•  We’ll  introduce  new  constant  symbols  d0,  di, . . . ,  d#  to  denote  levels  of  task 
decomposition,  and  a  predicate  symbol  level  so  that  the  atom  level  (d ; )  can 
be  used  to  mean  that  the  current  level  of  task  decomposition  is  d;.  We  give 
a  special  meaning  to  the  constant  symbol  d0:  it  marks  the  successful  end  of 
method  decomposition  process. 

•  To  specify  a  total  ordering  of  the  constant  symbols,  we  will  put  new  atoms 
next(di,d2),  next(d2,d3),  . . . ,  next(d#_i,  d#)  into  the  initial  state. 

In  addition,  for  each  method  m(x i, . . . ,  Xk)  and  task  t(yi, . . . ,  ?/, )  we  will  introduce 
new  atoms  dom(xi, . . . ,  Xk)  and  dot(yi, . . . ,  yj). 

4.2.2  Translating  Operators 

Let  o  be  any  operator  in  O,  and  suppose  name(o)  =  o(x i, . . . ,  xn ),  precond(o)  = 
{pi, . . .  ,pj}  and  effects(o)  =  {e3 , . . . ,  e^}.  If  o  is  not  mentioned  in  M  (whence  M  is 
not  O-complete),  then  trans(o )  =  o.  Otherwise  trans(o)  is  the  following  operator 
o',  which  is  like  o  except  that  it  is  applicable  only  when  doQ  is  true,  and  it  decrements 
the  counter: 


name(o')  =  o\x i, . . . ,  xn) 
precond(o/)  =  {do0(xi, . .  .,xn),p1, . . .  ,pj, 
level (v),  next (u,  v)} 

effects(o')  =  {^doD(a;i, . . . ,  xn ),  -ilevel(u), 
level (u),e1; . . .  ,ekj 
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We  define  transiO)  =  { trans(o )  |  o  G  O}. 

4.2.3  Translating  Methods 

Let  m  be  any  method  in  M,  and  suppose  name(m)  =  m(x i, . . . ,  xn),  task(m)  = 
t(yi, . . . ,  yjt),  and  precond(m)  =  {p\, . . .  ,Pjm}.  There  are  two  cases: 

Case  1:  subtasks(m)  =  0  (i.e.,  m  specifies  no  subtasks  for  t).  Then  transom) 
is  the  operator  m!  defined  as  follows: 

nam e(m')  =  m'(x i, . . . ,  xn) 
precond(m')  =  (dOf(j/i, . . . ,  yjt),Pi, . . .  ,pjy 
level(u),  next('u,  u)} 

effectsirn/)  =  {-idot(yi, . . .  ,  — 'level (v) ,  level(u)} 

Case  2:  subtasks(m)  =  {t\, . . .  ,tk}  for  k  >  1.  Then  transom)  is  the  set  of 
planning  operators  {m'0, . . . ,  rn'k }  defined  below,  where  m'0  is  an  operator  that  checks 
whether  m  is  applicable,  and  rn\ , . . .  ,m'k  are  operators  that  correspond  to  calling 
m’s  subtasks.  The  definition  of  m'0  is 

name(mg)  =  m'0(xi, . . . ,  xn) 
precond(mg)  =  (dot(j/i, . . .  ,yjt),pu  . .  .,pjm,  level  (u)} 
effects(mg)  =  {— «dot(j/i, . . . ,  yjt),  domi(xi, . .  .  7xn,v)} 

Intuitively,  mg’s  preconditions  say  that  t  is  the  current  task  and  that  m’s  precon¬ 
ditions  hold;  and  mg’s  effect  doTOl  makes  it  possible  to  apply  the  planning  operator 
rri\  that  corresponds  to  m’s  first  subtask. 
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For  i  —  1, , . . ,  k  —  1,.  iif  m’s  ith  subtask  is  U{yn , . . . ,  then  m'  is  defined  as 
follows: 

name(m-)  —  m'^x i, ...  ,xn) 

precond(m-)  =  {doTOi(xi, . . .  ,xn,v),  level  (v),  next(w,  w)} 
effects  (m-)  =  {^dom.  (xi, xn,  v),  ->  level  (v),  level  (w), 
doti(j/ii,  •  •  • ,  VijJ,  dom;+1  (xi, . . , ,  xn,  u)} 

The  operator  m'k ,  which  corresponds  to  m’s  last  subtask  is  like  m'  but  omits  the 
effects  -i level (v)  and  level (w). 

We  define  trans(M)  =  \JmeMtrans(m). 

4.2.4  Translating  Planning  Problems 

Finally,  we  define  trans(P)  =  (s'0,g,O'),  where 

s'0  =  s0U{next(do,di),...,next(dfc_i,dfc), 
level (d i) ,  doio(ci, . . .  ,cn)}; 

O'  =  trans(0)  U  trans(M). 


4.3  Properties 

The  theorems  in  this  section  establish  the  correctness  and  computational  com¬ 
plexity  of  our  translation  scheme. 

Theorem  4.1  Let  P  =  (s0,  (h, . . .  ,tk),0,M )  (where  k  >  0)  be  any  TSTN planning 
problem.  Let  A  —  {all  derivations  of  solutions  for  P},  and  U  =  {all  solutions  for 
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trans(P)} .  If  M  is  O -complete,  then  there  is  a  one-to-one  correspondence  that  maps 
A  onto  II. 

Sketch  of  proof.  We  need  to  define  a  mapping  F  :  A  — >•  II  and  show  that  F  is 
one-to-one  and  onto.  Below  we  define  F;  the  proof  that  it  is  one-to-one  and  onto 
can  be  done  straightforwardly  by  induction. 

Let  7T  be  a  solution  for  P  with  derivation  6.  Recall  that  6  is  the  sequence  of 
the  actions  and  method  instances  used  to  produce  7 r,  in  the  order  that  they  were 
applied.  In  particular,  5  is  a  concatenation  of  subsequences  5i, ...  ,5k  corresponding 
to  ti, . . . ,  tk .  We  will  let  F(5)  =  F(5i)  •  . . .  •  F(5k),  where  •  denotes  concatenation, 
and  where  each  F(5j)  is  defined  recursively  as  follows: 

If  5-j  is  empty,  then  F{5f)  also  is  empty.  If  5t  is  nonempty  (i.e.,  5t  =  {an, . . . ,  a;*,)), 
then  let  5'  =  (ai2, . . . ,  a^).  There  are  three  cases: 

1.  If  an  is  an  action,  then  F{5f)  =  transfan )  •  F{$i)- 

2.  If  an  is  a  substitution  instance  m6  of  a  method  m  with  substitution  6,  and 
subtasks(m)  is  empty,  then  F{5f)  =  rn'9  •  F{5,i),  where  m'  is  as  in  Case  1  of 
Section  3. 

3.  If  an  is  a  substitution  instance  rrid  of  a  method  m  and  subtasks(m)  is  nonempty 
(i.e.,  subtasks(m)  =  (t\ , . . . ,  f')  for  some  j  >0),  then  5[  is  the  concatenation 
of  subsequences  5[x, . ,  5f  produced  by  decomposing  t\, ...  ,t'rj,  respectively. 
In  this  case, 

F(5i)  =  m'o0  •  m[9  •  F{5'a)  •  . . .  •  m'  d  •  F{5'ij), 
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where  m^, ,  m'-  are  as  in  Case  2  of  Section  3.  □ 

Corollary  4.2  In  Theorem  4-1,  if  M  is  not  O-complete,  then  the  mapping  F  is 
one-to-one  but  not  necessarily  onto. 

Proof  Sketch.  If  M  is  not  O-complete,  then  there  is  at  least  one  operator  o  G  O 
that  is  not  mentioned  in  M .  Consequently,  no  instance  of  o  will  appear  in  any 
solution  for  P,  nor  in  A,  hence  no  instance  of  trans(o)  will  appear  in  {F(8)  \  6  G  A}. 
But  instances  of  trans(o )  can  appear  in  solutions  to  trans(P),  in  which  case  F  is 
no  longer  onto.  □ 

Theorem  4.3  The  time  and  space  complexity  of  computing  trans(P)  are  both  0(|P|  + 
H). 

Proof  Sketch.  For  each  o  G  O,  trans(o )  is  a  single  operator  that  is  computed  by  a 
linear-time  scan  of  o,  and  it  can  be  seen  by  inspection  that  the  size  of  that  operator 
is  0(|o|).  Suppose  there  are  no  non-tail  recursive  methods  in  M.  This  means  that 
H  =  0  in  this  case.  For  each  m  G  M,  trans(m)  is  a  set  of  methods  that  can  be 
produced  by  a  linear-time  scan  of  m,  and  it  can  be  seen  by  inspection  that  the  set 
of  methods  has  size  0(\m\).  If  there  is  a  non-tail  recursive  method  in  M,  then  H  is 
given  as  input  and  it  is  a  fixed  number.  Thus,  the  theorem  follows.  □ 

4.4  Implementation  and  Experiments 

We  implemented  an  algorithm  that  uses  our  translation  technique  to  translate 
TSTN  domain  descriptions  into  PDDL,  and  did  an  experimental  investigation  of 
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the  following  question: 


In  domains  that  are  hard  for  a  classical  planner,  how  much  can  its  per¬ 
formance  be  improved  by  PDDL  translations  of  partial  HTN  knowledge? 

For  the  classical  planner,  we  used  FF  [Hoffmann  and  Nebel,  2001].  FF  is  perhaps  one 
of  the  most  influential  classical  planners  available;  many  recent  classical  planning 
algorithms  either  directly  depends  on  generalizations  of  FF  or  they  incorporate  the 
core  ideas  of  FF  in  their  systems. 

For  the  planning  domains,  we  chose  three  planning  domains  for  which  we 
wrote  simple  HTN  domain  descriptions  with  varying  amounts  of  incompleteness: 
the  Blocks  World,  the  Towers  of  Hanoi  problem,  and  a  transportation  domain  called 
the  Office  Delivery  domain. 

The  source  code  for  our  translation  technique  and  the  HTN  method  descrip¬ 
tions  of  the  three  planning  domains  described  below  are  available  at  http://www. 
cs . umd . edu/ projects/planning/ data/ alf ord09translat ing/. 

4.4.1  Towers  of  Hanoi 

The  Towers  of  Hanoi  problem  causes  problems  for  many  classical  planners 
because  of  its  combinatorial  nature.  On  the  other  hand,  it  is  almost  trivially  easy 
to  write  a  set  of  HTN  methods  to  solve  the  problem  without  any  backtracking.  The 
methods  say  basically  the  following: 

•  Method  to  move  a  disk: 

precond:  the  smallest  disk  wasn’t  the  last  one  moved 
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subtask:  move  the  smallest  disk  clockwise. 


•  Method  to  move  a  disk: 

precond:  the  smallest  disk  was  the  last  one  moved 
subtask:  move  the  other  disk. 

Note  that  in  a  tower  of  a  Towers  of  Hanoi  problem,  the  largest  disk  is  always  at  the 
bottom  of  the  tower  and  no  disk  can  be  place  on  a  smaller  disk  -  i.e.,  the  disks  in 
a  tower  are  in  the  increasing  order  by  their  sizes  with  the  smallest  is  always  at  the 
top.  Thus,  whether  the  smallest  disk  was  the  last  one  moved  can  be  checked  in  the 
above  methods  by  examining  the  towers  to  the  left  or  to  the  right  of  a  tower. 

The  methods  above  provide  an  almost-complete  solution  to  the  Towers  of 
Hanoi  problem,  except  that  the  second  method  doesn’t  say  where  to  move  the  disk. 
To  use  the  PDDL  translation,  FF  must  figure  out  for  itself  that  there  is  only  one 
place  the  disk  can  be  moved. 

Below,  “FF-Plain”  refers  to  FF  using  the  ordinary  classical-planning  defini¬ 
tion  of  the  Towers  of  Hanoi  domain,  and  “FF-HTN”  refers  to  FF  using  the  PDDL 
translations  of  the  HTNs  described  above.  We  varied  the  number  n  of  disks  from  3 
to  14.  For  each  value  of  n,  we  ran  FF-HTN  and  FF-Plain  each  100  times,  averaging 
the  running  times.  The  reason  for  the  multiple  runs  is  because  FF  makes  some 
random  choices  during  each  run  that  make  its  running  time  vary  from  one  run  to 
another.  Fig.  4.1  shows  the  results. 

For  FF-Plain  at  14  rings  (the  *  in  Fig.  4.1),  two  runs  took  longer  than  2  hours 
(our  time  limit  per  problem)  to  finish.  We  counted  these  runs  as  2  hours  each, 
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Figure  4.1:  FF’s  CPU  time  in  the  Towers  of  Hanoi  domain,  with  and  without  the 
translated  domain  knowledge.  Each  data  point  is  FF’s  average  CPU  time  on  100 
runs.  The  asterisk  is  explained  in  the  text. 
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Number  of  Blocks 


Figure  4.2:  FF’s  CPU  time  in  the  Blocks  World,  with  and  without  the  translated 
domain  knowledge.  Each  data  point  is  FF’s  average  CPU  time  on  100  randomly 
generated  planning  problems.  The  asterisks  are  explained  in  the  text. 
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and  averaged  them  with  the  other  98  runs;  hence  the  data  point  for  14  rings  makes 
FF-Plain’s  performance  look  better  than  it  actually  was. 

As  shown  in  Fig.  4.1,  FF-Plain’s  running  times  grew  much  faster  than  FF- 
HTN’s  did.  With  14  disks,  FF-HTN  was  about  2  orders  of  magnitude  faster  than 
FF-Plain. 

4.4.2  Blocks  World 

The  Blocks  World  has  previously  been  shown  to  pose  some  difhculities  for  FF. 
Complete  HTN  domain  descriptions  can  work  very  efficiently  [Ghallab  et  ah,  2004], 
but  are  somewhat  complicated.  To  see  how  well  FF  could  do  with  some  simple  and 
partial  HTN  knowledge,  we  wrote  HTN  methods  that  said  basically  the  following: 

•  Method  to  move  a  block: 

precond:  the  block  is  not  in  its  final  position 
subtasks:  pick  up  the  block;  put  it  in  its  final  position. 

•  Method  to  move  a  block: 

precond:  the  block  is  not  in  its  final  position 
subtasks:  pick  up  the  block;  put  it  on  the  table. 

At  each  point  in  the  planning  process,  both  of  the  methods  are  applicable.  To  use 
the  PDDL  translation  of  them,  FF  must  use  its  heuristics  to  choose  which  of  them 
to  use. 

Below,  “FF-Plain”  refers  to  FF  using  the  ordinary  classical-planning  definition 
of  the  Blocks  World,  and  “FF-HTN”  refers  to  FF  with  the  PDDL  translations  of 
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Figure  4.3:  FF’s  CPU  time  in  the  Office  domain,  with  and  without  the  translated 
domain  knowledge.  In  the  graph  at  left,  the  number  of  packages  is  fixed  at  40  and 
the  number  of  rooms  varies.  In  the  graph  at  right,  the  number  of  rooms  is  fixed  at 
40  and  the  number  of  packages  varies.  Each  data  point  is  FF’s  average  CPU  time 
on  100  randomly  generated  planning  problems. 


the  methods  described  above.  We  ran  both  FF-HTN  and  FF-Plain  on  100  randomly 
generated  n- block  problems  for  each  of  n  =  5, 10, 15, ... ,  90,  giving  a  total  of  1800 
Blocks  World  problems.  Fig.  4.2  shows  the  results. 

As  before,  we  gave  FF  a  2-hour  time  limit  for  each  run.  At  data  points  where 
all  100  runs  took  less  than  2  hours  each,  the  data  point  was  the  average  time  per 
run.  At  data  points  where  3  or  fewer  of  the  100  runs  failed  to  finish  within  2  hours, 
we  counted  each  failure  as  2  hours  when  computing  the  average,  and  marked  the 
data  point  with  an  asterisk.  In  all  of  the  other  cases,  a  large  number  of  the  100  runs 
failed  to  finish  with  2  hours,  so  we  omitted  those  data  points. 

As  shown  in  Fig.  4.2,  FF-Plain  could  not  solve  problems  larger  than  25  blocks, 
but  FF-HTN  could  solve  problems  up  to  85  blocks.  At  25  blocks,  FF-HTN  was 
about  4.2  orders  of  magnitude  faster  than  FF-Plain. 
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4.4.3  Office  Delivery 


This  is  a  transportation  domain  in  which  a  robot  needs  to  pick  np  and  deliver 
packages  in  a  building.  It  is  similar  to  the  well-known  Robot  Navigation  Domain 
[Kabanza  et  ah,  1997],  with  the  following  differences:  (1)  the  problem  is  determin¬ 
istic,  (2)  there  is  a  variable  number  of  rooms,  and  (3)  some  of  the  rooms  can  be 
quite  far  from  the  hallway  (hence  to  get  to  a  room  r,  the  robot  may  need  to  go 
through  many  other  rooms).  For  this  domain,  we  wrote  a  very  incomplete  set  of 
HTN  methods: 

•  Method  to  move  all  remaining  packages: 

precond:  there  is  a  package  that’s  not  at  its  destination 
subtasks:  pick  up  the  package;  put  it  in  its  final  location; 
move  all  remaining  packages. 

Above,  we  omitted  (1)  how  to  get  to  the  package’s  location  in  order  to  pick  it  up, 
and  (2)  how  to  take  the  package  to  its  destination.  To  use  the  PDDL  translation  of 
the  method,  the  planner  must  figure  out  those  things  for  itself. 

Fig.  4.3  shows  the  results  of  our  Office  Delivery  experiments.  For  the  graph 
on  the  left,  we  fixed  the  number  of  packages  at  40  and  varied  the  number  n  of  office 
rooms  from  10  to  90;  and  for  the  graph  on  the  right  we  fixed  the  number  of  rooms 
at  40  and  varied  the  number  k  of  packages  from  10  to  90.  For  each  combination 
of  n  and  k  we  ran  FF  on  100  randomly  generated  problems,  giving  a  total  of  1700 
problems. 
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As  shown  in  Fig.  4.3,  FF-Plain’s  running  time  increased  much  faster  than  FF- 
HTN’s.  On  the  largest  problems  (90  rooms  and  90  packages),  FF-HTN  was  faster 
than  FF-Plain  by  about  2.8  and  1.9  orders  of  magnitude,  respectively. 

4.5  Discussion 

This  chapter  shows  that  ffTN  planning  knowledge,  if  it  satisfies  the  restrictions 
described  in  Section  2.2,  can  easily  be  translated  into  a  form  usable  by  domain- 
independent  PDDL  planners. 

In  our  experiments  with  FF,  PDDL  translations  of  small  amounts  of  HTN 
planning  knowledge  improved  FF’s  performance  by  several  orders  of  magnitude. 
This  occurred  even  though  the  HTN  knowledge  was  incomplete,  i.e.,  it  omitted  some 
of  the  knowledge  that  an  HTN  planner  would  need.  In  places  where  the  knowledge 
was  missing,  FF  simply  used  its  ordinary  planning  heuristics. 

FF’s  ability  to  augment  the  translated  HTN  knowledge  with  its  own  heuristics 
show  that  even  if  a  full-semantics  delete-relaxation  HTN  heuristic  is  out  of  reach, 
relaxed-semantics  heuristics  can  effectively  guide  the  search  to  quality  plans.  Delete 
relaxation  heuristics  with  semantics  that  more  closely  match  that  of  regular  HTN 
planning  should  be  more  informed  (and  thus  more  effective).  Since  forward-search 
planners  such  as  FF  and  LAMA  are  guaranteed  to  terminate,  running  translated 
HTN  problems  under  FF  and  LAMA  is  a  decision  procedure  for  totally-ordered 
<r-strati£able  problems.  Although  the  experiments  are  not  enough  to  show  out  the 
impact  of  the  duplicate  detection  required  to  maintain  termination,  they  at  least 


84 


show  it  is  not  an  impediment  to  effective  search. 
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Chapter  5:  Conclusions 


This  research  contributes  to  the  understanding  of  search  in  hierarchical  plan¬ 
ning  systems,  both  in  how  best  to  organize  the  search  space  and  the  restrictions  on 
developing  domain  independent  heuristics.  Guaranteed  termination  also  opens  up 
the  possibility  of  adapting  a  number  of  techniques  from  classical  planning  to  HTN 
planning,  such  planning-as- verification  [Albarghouthi  et  ah,  2009],  counter  exam¬ 
ple  generation  [Goldman  et  ah,  2012],  and  policy  generation  for  non-deterministic 
problems  [Kuter  et  ah,  2008]. 

Specifically,  this  dissertation  provides: 

•  Four  algorithms  for  HTN  planning  (DHTN,  PHTN,  TODHTN,  and  TOPHTN) 
and  four  corresponding  classes  of  syntactically-identihablc  HTN  problems  for 
which  those  algorithms  are  proven  to  terminate  (<i-stratifiablc,  <r-strati£able, 
<i-ordered,  and  <r-ordered,  respectively). 

•  Upper  and  lower  complexity  bounds  for  each  of  these  identifiable  sets,  includ¬ 
ing  new  bounds  on  previously  known  decidable  problem  sets. 

•  An  analysis  of  delete-free  HTN  planning,  showing  that  almost  all  delete-free 
HTN  planning  is  NP-complete.  This  illuminates  one  of  the  barriers  to  provid- 


ing  domain-independent  HTN  heuristics. 


•  A  method  for  HTN  planning  via  translation  to  PDDL  for  any  totally-ordered 

<r-strati£able  problem. 

A  caveat  to  the  above  work  is  that  all  the  algorithms  were  analyzed  over 
propositional  HTN  planning,  while  most  planning  implementations  use  function-free 
fragments  of  first-order  logic  (though  many  planners  create  a  ground  propositional 
domain  as  a  pre-processing  step).  For  the  most  part,  this  just  provides  an  expo¬ 
nential  bump  in  the  complexity  bounds  [Erol  et  al.,  1995,  1996],  but  we  have  not 
formally  shown  that. 

Although  this  thesis  provides  no  direct  implementation  of  these  algorithms 
nor  a  specialized  HTN  heuristic,  when  used  with  a  classical  planner,  the  HTN- 
to-PDDL  translation  does  provide  heuristic,  terminating  search  for  totally-ordered 
<r-stratifiable  problems.  Not  only  does  this  provide  a  practical  benefit  for  those 
who  need  heuristic  HTN  search,  it  provides  a  baseline  from  which  to  evaluate  future 
HTN-specihc  domain-independent  heuristics. 

This  leaves  a  clear  future  work:  the  implementation  and  evaluation  of  TOPHTN 
described  in  Section  2.3,  and  the  development  and  evaluation  of  principled  delete- 
free  based  HTN  heuristics. 

Guaranteed  termination  paired  with  heuristic  search  will  allow  for  search  when 
the  solvability  of  the  problem  is  in  question,  such  as  during  debugging,  or  for  policy 
generation  for  non-deterministic  problems.  Even  when  the  problem  is  known  (or 
assumed)  to  be  solvable,  current  HTN  planners  such  as  SHOP2  Nan  et  al.  [2003] 
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must  default  to  depth-first  search  to  effectively  find  plans.  This  requires  the  domain 
author  to  know  what  paths  the  planner  will  take,  and  ensure  that  these  paths 
are  all  finite  or  end  in  a  solution.  Otherwise,  if  the  path  the  planner  chooses  is 
infinite,  the  planner  will  fail  to  return  a  solution  even  when  one  exists.  In  contrast, 
if  the  problem  <r-ordered  (the  broadest  of  the  syntactic  classes  presented  here),  a 
TOPHTN  implementation  is  guaranteed  to  terminate  regardless  of  how  it  orders  its 


search  space. 
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